Add git repository aliasing and quick module declaration to kdesrc-build.

This commit adds two options (not yet documented in the DocBook):

1. git-repository-base, which defines a new repository alias (kde-git in the sample file)

In order to actually use this repository alias, you can use a new type of module
(unimagineatively called a "module-set") like so:

global
 ...
end global

module-set $repo_name
  use-modules <list of git modules e.g. akonadi attica soprano k3b etc.>
end module-set

No other options are supported in a module-set at this point. kdesrc-build treats a
module-set as if you'd simply manually declared a new module for each item you passed
to use-modules, and sets the "repository" option for each new module to the base URL
(as determined by repo_name), with the module name tacked on.

kdesrc-build will filter any trailing .git off of the virtual module name.

If you still want to add options for a module, you can simply manually declare a module
of that name after the module-set, and choose options there (it won't change the build
order though).

Hopefully this makes it that much easier to use the plethora of git modules that have
been getting successfully transferred over to git.kde.org. PLEASE let me know if I've
broken something here or otherwise improperly implemented it. CC'ing dfaure since I
figure he'll be interested and/or have feedback.

CCMAIL:dfaure@kde.org

svn path=/trunk/KDE/kdesdk/scripts/kdesrc-build; revision=1204410
wilder
Michael Pyne 16 years ago
parent 8f88f90e38
commit 5092021c2b
  1. 290
      kdesrc-build
  2. 70
      kdesrc-buildrc-sample
  3. 35
      kdesrc-buildrc.xml

@ -95,7 +95,9 @@ our %package_opts = (
"do-not-compile" => "",
"email-address" => "",
"email-on-compile-error" => "",
"install-after-build" => 1, # Default to true
"git-repository-base" => {}, # Base path template for use multiple times.
"use-modules" => "",
"install-after-build" => 1, # Default to true
"kdedir" => "$ENV{HOME}/kde",
"kde-languages" => "",
"libpath" => "",
@ -2921,6 +2923,97 @@ sub setup_default_modules()
}
}
# Reads a "line" from a file. This line is stripped of comments and extraneous
# whitespace. Also, backslash-continued multiple lines are merged into a single
# line.
#
# First parameter is the reference to the filehandle to read from.
# Returns the text of the line.
sub read_line
{
my $fh = shift;
while(<$fh>) {
# Remove trailing newline
chomp;
# Replace \ followed by optional space at EOL and try again.
if(s/\\\s*$//)
{
$_ .= <$fh>;
redo unless eof($fh);
}
s/#.*$//; # Remove comments
next if /^\s*$/; # Skip blank lines
return $_;
}
return undef;
}
# Takes an input line, and extracts it into an option name, and simplified
# value. The value has "false" converted to 0, white space simplified (like in
# Qt), and tildes (~) in what appear to be path-like entries are converted to
# the home directory path.
#
# First parameter is the input line.
# Return value is (optionname, option-value)
sub split_option_value
{
my $input = shift;
my $optionRE = qr/\$\{([a-zA-Z0-9-]+)\}/;
# The option is the first word, followed by the
# flags on the rest of the line. The interpretation
# of the flags is dependant on the option.
my ($option, $value) = ($input =~ /^\s* # Find all spaces
([-\w]+) # First match, alphanumeric, -, and _
# (?: ) means non-capturing group, so (.*) is $value
# So, skip spaces and pick up the rest of the line.
(?:\s+(.*))?$/x);
$value = "" unless defined $value;
# Simplify this.
$value =~ s/\s+$//;
$value =~ s/^\s+//;
$value =~ s/\s+/ /;
# Check for false keyword and convert it to Perl false.
$value = 0 if lc($value) eq 'false';
# Replace reference to global option with their value.
# The regex basically just matches ${option-name}.
my ($sub_var_name) = ($value =~ $optionRE);
while ($sub_var_name)
{
my $sub_var_value = get_option('global', $sub_var_name);
if(not $sub_var_value)
{
my $line_no = $.;
warning " *\n * WARNING: $sub_var_name is not set at line y[$line_no]\n *";
$sub_var_value = '';
}
debug "Substituting \${$sub_var_name} with $sub_var_value";
$value =~ s/\${$sub_var_name}/$sub_var_value/g;
# Replace other references as well. Keep this RE up to date with
# the other one.
($sub_var_name) = ($value =~ $optionRE);
}
# Replace tildes with home directory.
1 while ($value =~ s"(^|:|=)~/"$1$ENV{'HOME'}/");
return ($option, $value);
}
# Reads in the options from the config file and adds them to the option store.
# The first parameter is a reference to the file handle to read from.
# The second parameter is 'global' if we're reading the global section, or
@ -2938,20 +3031,8 @@ sub parse_module
}
# Read in each option
while (<$fh>)
while (read_line($fh))
{
# Handle line continuation
chomp;
if(s/\\\s*$//) # Replace \ followed by optional space at EOL and try again.
{
$_ .= <$fh>;
redo unless eof($fh);
}
s/#.*$//; # Remove comments
next if /^\s*$/; # Skip blank lines
if($module eq 'global')
{
last if /^end\s+global/; # Stop
@ -2969,56 +3050,90 @@ sub parse_module
exit 1; # exit instead of finish() since we're not fully setup yet.
}
# The option is the first word, followed by the
# flags on the rest of the line. The interpretation
# of the flags is dependant on the option.
my ($option, $value) = /^\s* # Find all spaces
([-\w]+) # First match, alphanumeric, -, and _
# (?: ) means non-capturing group, so (.*) is $value
# So, skip spaces and pick up the rest of the line.
(?:\s+(.*))?$/x;
my ($option, $value) = split_option_value($_);
$value = "" unless defined $value;
# Handle special options.
if ($module eq 'global' && $option eq 'git-repository-base') {
# This will be a hash reference instead of a scalar
my ($repo, $url) = ($value =~ /^([a-zA-Z0-9_-]+)\s+(.+)$/);
$value = get_option('global', $option) || { };
# Simplify this.
$value =~ s/\s+$//;
$value =~ s/^\s+//;
$value =~ s/\s+/ /;
if (!$repo || !$url) {
error <<"EOF";
The y[git-repository-base] option at y[b[$rcfile:$.]
requires a repository name and URL.
# Check for false keyword and convert it to Perl false.
$value = 0 if lc($value) eq 'false';
e.g. git-repository base kde git://anongit.kde.org/
# Replace reference to global option with their value.
# The regex basically just matches ${option-name}. Keep the RE up
# to date with the same one below.
Use this in a "module-set" group:
my ($sub_var_name) = ($value =~ m/\$\{([a-zA-Z0-9-]+)\}/);
while ($sub_var_name)
{
my $sub_var_value = get_option('global', $sub_var_name);
e.g.
module-set kde
use-modules automoc akonadi soprano attica
end module-set
EOF
exit 1;
}
if(not $sub_var_value)
{
my $line_no = $.;
warning " *\n * WARNING: $sub_var_name is not set at line y[$line_no]\n *";
$value->{$repo} = $url;
}
$sub_var_value = '';
}
set_option($module, $option, $value);
}
}
# Reads in a "moduleset".
#
# First parameter is the filehandle to the config file to read from.
# Second parameter is the name of the moduleset, which is really the name
# of the base repository to use.
# Returns the expanded list of module names to include.
sub parse_moduleset
{
my ($fh, $repoName) = @_;
my $repoSet = get_option('global', 'git-repository-base');
my @modules;
if (not exists $repoSet->{$repoName}) {
error "There is no repository assigned to y[b[$repoName] when assigning";
error "a moduleset on line $. of $rcfile";
exit 1;
}
debug "Substituting \${$sub_var_name} with $sub_var_value";
while(read_line($fh)) {
return @modules if /^end\s+module(-?set)?$/;
$value =~ s/\${$sub_var_name}/$sub_var_value/g;
my ($option, $value) = split_option_value($_);
# Replace other references as well. Keep this RE up to date with
# the other one.
($sub_var_name) = $value =~ m/\$\{([a-zA-Z0-9-]+)\}/;
if ($option eq 'use-modules') {
@modules = split(' ', $value);
}
else {
warning "Ignoring option $option on line $. of $rcfile";
next;
}
# Replace tildes with home directory.
1 while ($value =~ s"(^|:|=)~/"$1$ENV{'HOME'}/");
if (not @modules) {
error "No modules were selected for repository y[b[$repoName]";
error "in the y[use-modules] on line $. of $rcfile";
exit 1;
}
set_option($module, $option, $value);
# Setup default options for each module
for my $module (@modules) {
my $moduleName = $module;
$moduleName =~ s/\.git$//; # Remove trailing .git for module name
if (not defined $package_opts{$moduleName})
{
$package_opts{$moduleName} = default_module_options($moduleName);
}
set_option($moduleName, 'repository', $repoSet->{$repoName} . $module);
}
}
return @modules;
}
# This subroutine reads in the settings from the user's configuration
@ -3068,7 +3183,7 @@ EOM
return;
}
my ($option, $flags, $modulename);
my ($option, $modulename, %readModules);
# Read in global settings
while (<CONFIG>)
@ -3106,20 +3221,33 @@ EOM
# Now read in module settings
while (<CONFIG>)
{
s/#.*$//; # Remove comments
s/^\s*//; # Remove leading whitespace
s/#.*$//; # Remove comments
s/^\s*//; # Remove leading whitespace
next if (/^\s*$/); # Skip blank lines
my @addedModules = ();
# Get modulename (has dash, dots, slashes, or letters/numbers)
($modulename) = /^module\s+([-\/\.\w]+)\s*$/;
if (not $modulename)
{
warning "Invalid configuration file $rcfile!";
warning "Expecting a start of module section at line $..";
warning "Global settings will be retained.";
($modulename) = /^module-set\s+([-\/\.\w]+)$/;
if (not $modulename) {
error "Invalid configuration file $rcfile!";
error "Expecting a start of module section at line $. of $rcfile.";
error "Global settings will be retained.";
$modulename = 'null'; # Keep reading the module section though.
exit 1;
}
# A moduleset can give us more than one module to add.
@addedModules = parse_moduleset(\*CONFIG, $modulename);
}
else {
parse_module(\*CONFIG, $modulename);
push @addedModules, $modulename;
}
# Don't build default modules if user has their own wishes.
@ -3129,24 +3257,27 @@ EOM
@update_list = @build_list = ( );
}
parse_module(\*CONFIG, $modulename);
next if ($modulename eq 'null');
# Don't re-add duplicated modules (it may be possible to see a module
# declaration for a module already seen in a module-set)
@addedModules = grep { not exists $readModules{$_} } (@addedModules);
# Done reading options, add this module to the update list
push (@update_list, $modulename) unless exists $ignore_list{$modulename};
# Done reading options, add this module(s) to the update lists
push (@update_list, @addedModules);
push (@build_list, @addedModules);
# Add it to the build list, unless the build is only
# supposed to be done manually.
if (not get_option ($modulename, 'manual-build') and not exists $ignore_list{$modulename})
{
push (@build_list, $modulename);
}
# Add an entry for each module that has been read in so we don't
# add it later to update/build list.
@readModules{@addedModules} = 1;
}
close CONFIG;
delete $package_opts{'null'}; # Just in case.
# All modules and their options have been read, filter out modules not
# to update or build, based on the --ignore-modules option already present
# on the command line. manual-update and manual-build are also relevant,
# but handled in get_build_list or get_update_list.
@update_list = grep { not exists $ignore_list{$_} } (@update_list);
@build_list = grep { not exists $ignore_list{$_} } (@build_list);
# If the user doesn't ask to build any modules, build a default set.
# The good question is what exactly should be built, but oh well.
@ -4219,13 +4350,9 @@ sub get_update_list
push @update_list, $l10n; # Add l10n module to potential update list.
}
my @return_list;
for (@update_list)
{
push @return_list, $_ if not get_option($_, "manual-update");
}
return @return_list;
return grep {
not get_option($_, 'manual-update')
} (@update_list);
}
# Subroutine to get the list of KDE modules to build. Returned
@ -4307,13 +4434,10 @@ EOF
return @ARGV unless $#ARGV == -1;
my @list;
for (@build_list)
{
push @list, $_ unless get_option($_, 'manual-update');
}
return @list;
return grep {
not get_option($_, 'manual-update') and
not get_option($_, 'manual-build')
} (@build_list);
}
# Used to sort module names. 'global' always starts first, modules with /

@ -40,7 +40,14 @@ global
# but also see the qt-copy and kdesupport modules below, which have special
# requirements
# This is the Subversion server to download the KDE sources from. Developers:
# To use a single git repository to download multiple KDE source modules, you can
# define a "base repository", which you can use later with a "module-set" declaration
# and the "use-modules" option. This sets up the "kde-git" git repository, you can change
# the URL if you are a developer.
git-repository-base kde-git git://anongit.kde.org/
# git-repository-base kde-git git@git.kde.org: # Developers
# This is the Subversion server to download the rest of the KDE sources from. Developers:
# Don't forget to add your username to the URL if necessary!
# svn-server svn://anonsvn.kde.org/home/kde
@ -160,6 +167,23 @@ module qt-copy
branch master
end module
# Next is shown a "module-set", which is passed the name of the repository base (defined
# in the global section with git-repository-base). The only option accepted inside a
# module-set is the 'use-modules' option, which is simply passed a list of each module
# you want to build. For every single module passed, kdesrc-build will create the
# appropriate "module" option, with the right repository setting (base + module name).
#
# If you need to set more options, you can add a module <name> ... end module as normal
# after the module-set, but keep in mind if you do this that you won't change the order
# the modules are built in.
#
# Example:
module-set kde-git
# automoc is essential for building KDE's CMake-based modules, and so should be first
# in this module set (but after Qt, if you're using qt-copy)
use-modules automoc cagibi attica soprano polkit-qt-1
end module-set
# kdesupport contains taglib and QCA, and the Strigi library required for
# kdelibs in KDE 4. taglib is required for JuK, amarok, and the meta info
# reader for music files in Konqueror.
@ -173,31 +197,6 @@ module kdesupport
cmake-options -DWITH_ASF=TRUE -DWITH_MP4=TRUE
end module
# automoc is very needed for KDE's cmake support, install it before all KDE modules but after Qt 4.
module automoc
repository git://git.kde.org/automoc
end module
module cagibi
repository git://git.kde.org/cagibi
end module
module attica
# Read-only access
repository git://git.kde.org/attica
# Read-write access
# repository git@git.kde.org:attica
end module
module soprano
# Read-only access
repository git://git.kde.org/soprano
# Read-write access
# repository git@git.kde.org:soprano
end module
module polkit-qt-1
repository git://git.kde.org/polkit-qt-1
end module
# Phonon, the KDE multimedia interface, is required for KDE. It is present in
# Qt but the "official" development branch for it is on gitorious.org (it used
# to be in kdesupport, above).
@ -215,13 +214,11 @@ module kdelibs
# separate script in kdesdk/scripts to do that for you however.
end module
# akonadi is required for kdepimlibs
module akonadi
# Read-only access
repository git://git.kde.org/akonadi
# Read-write access
# repository git@git.kde.org:akonadi
end module
# akonadi is required for kdepimlibs. Added to this module-set so that it is next to
# kdepimlibs -- it could just as well be part of the earlier module-set.
module-set kde-git
use-modules akonadi
end module-set
# kdepimlibs contains required PIM (mail, instant messenger, etc.) basics
# that are needed by some other KDE modules. Should be built after kdelibs
@ -344,7 +341,12 @@ end module
# repository git@git.kde.org:amarok.git
#end module
# Includes the popular K3B cd/dvd burner and various media players.
# --- The K3B cd/dvd/etc. burner.
#module-set kde-git
# use-modules k3b
#end module-set
# Includes various media players.
#module extragear/multimedia
#end module

@ -3,7 +3,7 @@
<!-- This goes into $KDEDIR/share/apps/katepart/syntax, or
it can go to $KDEHOME/share/apps/katepart/syntax
-->
<language name="kdesrc-buildrc" version="0.5.1" kateversion="2.4" section="Configuration" extensions=".kdesrc-buildrc;kdesrc-buildrc;.kdesvn-buildrc;kdesvn-buildrc" author="Michael Pyne &lt;mpyne@kde.org&gt;" license="LGPL">
<language name="kdesrc-buildrc" version="0.5.2" kateversion="2.4" section="Configuration" extensions=".kdesrc-buildrc;kdesrc-buildrc;.kdesvn-buildrc;kdesvn-buildrc" author="Michael Pyne &lt;mpyne@kde.org&gt;" license="LGPL">
<highlighting>
<list name="bools">
@ -17,6 +17,11 @@
<item>set-env</item> <!-- Handled separately for special syntax highlighting. -->
</list>
<!-- List of options that can be present in a module-set -->
<list name="moduleSetOptions">
<item>use-modules</item>
</list>
<!-- These options should only have a boolean value passed to them. -->
<list name="boolOptions">
<item>async</item>
@ -54,6 +59,7 @@
<item>do-not-compile</item>
<item>email-address</item>
<item>email-on-compile-error</item>
<item>git-repository-base</item>
<item>kdedir</item>
<item>kde-languages</item>
<item>libpath</item>
@ -70,6 +76,7 @@
<item>source-dir</item>
<item>svn-server</item>
<item>tag</item>
<item>use-modules</item>
</list>
<contexts>
@ -77,9 +84,15 @@
<DetectSpaces/>
<RegExpr attribute="Comment" String="#.*$" context="#stay"/>
<RegExpr attribute="Global Declaration" context="Module Options" String="global" beginRegion="Global Options" firstNonSpace="true" />
<RegExpr attribute="Module Declaration" context="ModuleSet Decl" String="module-set\s+" firstNonSpace="true" />
<RegExpr attribute="Module Declaration" context="Module Decl" String="module\s+" firstNonSpace="true" />
</context>
<context name="ModuleSet Decl" attribute="Module Name" lineEndContext="ModuleSet Options">
<RegExpr String="[a-zA-Z0-9/-]*" context="#stay" beginRegion="ModuleSet Options"/>
<RegExpr attribute="Comment" String="#.*$" context="#stay"/>
</context>
<context name="Module Decl" attribute="Module Name" lineEndContext="Module Options">
<RegExpr String="[a-zA-Z0-9/-]*" context="#stay" beginRegion="Module Options"/>
<RegExpr attribute="Comment" String="#.*$" context="#stay"/>
@ -96,15 +109,27 @@
<RegExpr attribute="Module Declaration" context="Module" endRegion="Global Options" String="end global" firstNonSpace="true"/>
<!-- No kdesrc-build options start with digits or a dash. -->
<RegExpr attribute="Possible Error" context="Option Value" String="[0-9-][a-zA-Z0-9-]+" />
<RegExpr attribute="Unknown Option Name" context="Option Value" String="[a-zA-Z0-9-]+" />
<RegExpr attribute="Possible Error" context="Option Value" String="[0-9-][a-zA-Z0-9_-]+" />
<RegExpr attribute="Unknown Option Name" context="Option Value" String="[a-zA-Z0-9_-]+" />
</context>
<context name="ModuleSet Options" attribute="Normal Text" lineEndContext="#stay">
<DetectSpaces/>
<RegExpr attribute="Comment" String="#.*$" context="#stay"/>
<keyword attribute="Option Name" context="Option Value" String="moduleSetOptions"/>
<RegExpr attribute="Module Declaration" context="Module" endRegion="ModuleSet Options" String="end module(-?set)?$" firstNonSpace="true"/>
<!-- No kdesrc-build options start with digits or a dash. -->
<RegExpr attribute="Possible Error" context="Option Value" String="[0-9-][a-zA-Z0-9_-]+" />
<RegExpr attribute="Unknown Option Name" context="Option Value" String="[a-zA-Z0-9_-]+" />
</context>
<context name="Option Value" attribute="Normal Text" lineEndContext="#pop">
<RegExpr attribute="Comment" String="#.*$" />
<DetectSpaces/>
<RegExpr attribute="Variable" String="\$\{[a-zA-Z0-9-]+\}"/>
<RegExpr attribute="Option Value" String="[a-zA-Z:0-9,./+!=@-]*" context="#stay" />
<RegExpr attribute="Variable" String="\$\{[a-zA-Z0-9_-]+\}"/>
<RegExpr attribute="Option Value" String="[a-zA-Z:0-9,./+!=@_-]*" context="#stay" />
<LineContinue context="#stay"/>
<AnyChar attribute="Option Value" String="\\"/>
</context>

Loading…
Cancel
Save