JSON::PP is included in core Perl so one less dependency to look after,
and it's fast enough (for what we need) that there's really no point to
making it possible to look for JSON::XS.
This option can be used to setup the user.name and user.email git-config
options within each new git repository cloned or installed from
snapshot, including kde-projects repositories. This can be useful for
KDE developers with multiple Git identities that they use for
development.
Module settings override global settings so you could e.g. have a global
git-user option for KDE modules and then override specific modules or
module-sets with a different git-user setting.
This setting is only applied during the initial git-clone process
though... if there's interest in having kdesrc-build re-apply settings
later that's possible too.
BUG:352478
FIXED-IN:15.09
When the initrepo script fails the extracted snapshot directory (the
srcdir) is deleted, but it's also the current working directory, which
breaks git when doing the fallback clone. Fix by switching to an
existing directory.
CCMAIL:faure@kde.org
Perl core includes a "HTTP::Tiny" module that does what little we need,
and has had it since 5.14 or so, so prefer that instead and remove the
requirement to install libwww-perl.
This also required removing the one minor instance of the URI module,
which is not technically in Perl core but was installed due to LWP.
Instead we just use a regexp 'blessed' by the URI module author.
This is a 2-part change:
1. Always download needed metadata, even in --pretend mode. This means
that even --pretend won't stop kdesrc-build from trying to hit the
network, but --pretend was always intended to let you test command lines
to ensure you got the right build list before you wasted hours trying to
build something. If you don't want kdesrc-build to touch the network
ever, don't run it. :)
2. Check for JSON decoding errors specifically (since it may happen that
the JSON is corrupted on download somehow, or accidentally broken
upstream).
This should allow "kdesrc-build -p --print-modules", or even
"kdesrc-build -p" to work even as the very first command ever run.
BUG:340481
FIXED-IN:1.16
There seems to be nothing better than checking for the existence of
files or directories under .git at this point (though it's possible to
decode git-status --porcelain output without too much trouble in the
case of in-progress merges).
This should hopefully help avoid kdesrc-build trying to update a module
that is being worked on.
BUG:334324
FIXED-IN:1.16
Bug 341159 complains about kdesrc-build-setup not setting up the kde:
git prefix, but kdesrc-build-setup itself never even calls git, it just
creates a .kdesrc-buildrc and exits.
However the currently recommended way to start out kdesrc-build is to
run "kdesrc-build --metadata-only" for the first build to setup
directories and download required data, while still letting a subsequent
"kdesrc-build --pretend" do something useful. kdesrc-build didn't setup
the kde: prefix in time to support --metadata-only, and I'm willing to
bet that's what the user was encountering here.
So, setup the git configuration before the first time we can use git.
BUG:341159
FIXED-IN:1.16
Fixes bug 342806, where a kdesrc-build user notes that "Cloning"
messages are indented while "Updating" messages are not, which can be
confusing.
BUG:342806
FIXED-IN:1.16
We use a mix of error codes and raw exceptions in the git updater.
Fix this to use the exception objects used elsewhere (and which give
better error messages).
This also simplifies the code by removing error code returns from some
internal functions whose return value is then just converted into an
exception anyways. Easier just to throw the exception directly and
simplify the higher-level functions.
The one nuance here is that installing a git snapshot can fail yet that
should not cause an exception, we'd just fallback to doing a normal
git-clone.
The recent port to Getopt::Long broke the ability to use --foo=bar type
of options (where <foo> is a global option name known to the build
context). It also replaced the --module,foo=bar options with
--set-module-option-value=module,foo,bar.
While it's possible to use the latter syntax for global options (by
using 'global' as the module name) that is undocumented and may not work
forever. And either way, --foo options did use to work, and I noted it
was broken in the refactor.
So now that someone else has noticed I've reimplemented the feature.
It's difficult to do entirely from within Getopt::Long since the
documentation for that module gives you essentially one catch-all, which
only supports *non-options*.
So what I've done instead is to have kdesrc-build dynamically introspect
its list of global options and flags and then add them to the list of
valid options we pass to Getopt::Long. This also should mean that we
don't need to pass 'pass_through' as an option to Getopt::Long anymore,
though we'll see.
This code path won't be used for options that already have a command
line option (such as --async/--no-async) so that is still a misfeature
compared to how it was before, but I guess it can't be perfect. =D
I've tried to ensure that "flags" support the 'false' value to mean
boolean false, but be careful to pass false or 0 if that's what you
mean... specifying --dont-foo without giving it a value defaults it to
true.
BUG:330386
This is a long-overdue change that allows for overloading the
stringification operator to produce better error messages.
BuildException was previously not an actual module since kdesrc-build
itself was just a single script file. Now that we allow modules on disk
there is no reason to leave the exception class as a figment of Perl's
imagination.
Before this we would try to apply branch-group to a module, it would
fail (since plain git modules don't *have* any kde-projects metadata),
and we would invariably select master instead of choosing based on the
plain git module's branch option.
Note that this still doesn't solve the problem of modules that are from
kde-projects but *don't* have branch-group metadata; if both module
branch-group and global branch are active for such a module, the
module-specific branch-group will be tried and resolve to 'master'
independent of the global branch setting.
Whether this is a feature or not is not yet certain. :)
Now the priority order for all of the various "I want to checkout this
commit" options are centralized in one spot, and both cloning and
updating obey the same options now (they didn't before).
We need a better name for this feature.
But basically, this allows for creating kde-project repository-wide
"branch groups". For instance, you could list the git branch to use for
the "stable Qt/KDE 4" release/maintenance branch, the "new development
Qt/KDE 4" development branch (typically master, but that has been
changing), and the "upcoming next major release" branch (for us, this
means 'frameworks' sometimes, means 'master' sometimes, other modules
may have different ideas).
By using the 'branch-group' option (which remains to be further
documented) you can say that you want to defer to the metadata about the
repository branches stored in
kde-build-metadata/logical-module-structure. kdesrc-build should even
track branches for you as they are updated by the Release Team, if you
simply want to hang out on stable but compile your own stuff.
Note that this support requires Perl's JSON module (which should come
with Perl 5.14 if I understand it right, but you probably want to
install JSON::XS from CPAN). JSON support is only required for this
feature, it is not a new overall dependency.
See also: http://community.kde.org/Infrastructure/Project_Metadata
The previous version worked on modules that had *no* user (a setup most
of mine had as the user was set in the ssh config).
For modules with a user set the trailing '@' was included in the uid
which make the check fail even when the module was correct. (Though I
had thought I had tested this case with other modules).
In the past some KDE SVN modules used the "svn external" feature to pull
in other SVN repositories. These svn-external links were on the *server*
side so the link URL had to use a single scheme.
This scheme was https, which caused quite some problems for anonsvn
users just trying to update from SVN being an unattended kdesrc-build
run, as the SSL cert for svn.kde.org was not signed by a well-known CA.
As a result kdesrc-build updated svn automatically with the expected KDE
SVN SSL signature to avoid blocking on the interactive warning.
This is no longer required. The modules that used svn externals have
pretty much moved onto git now. The new svn server does not even have an
https interface, so this is dead code now, and removed accordingly.
Though this only works for Git modules, it should at least fix the
annoying "OMG YOUR REPO MOVED" warnings that never get fixed when in
async mode (which is the default).
Previously you simply never saw the warning due to the now-fixed output
squelching, now we need to fix this for good as well.
For whatever dumb reason I coded the "ignored kde-projects modules"
handling (kde-build-metadata/build-script-ignore) by having the code
that reads in the list of ignored modules go and immediately update the
build context.
For obvious reasons this only works when the update is actually
performed, which means ignore information wasn't used in pretend mode.
We fix this by moving the build context update to the rest of the
metadata handling.
This should fix the big ugly error messages for this particular
scenario. The code in question is trying to determine a good branch name
that it can use, which means it need to verify that the requested branch
name *doesn't* exist. This would correspond to an exit code of 1 for the
given git command.
In pretend mode all safe_system() calls give an exit code of 0, so it
appears that there are no safe unused branch names.
Since even in pretend mode the source directory should exist in this
code path I switched to use a system() call (with the correct full
branch name to boot).
I'm not sure the other part of the bug is fixed (i.e. checking out a
local branch immediately after cloning or whatever dfaure was seeing) so
I'll leave open for now until I can do further testing.
CCBUG:311758
Skip the kde-build-metadata update phase if --no-src is passed, or if in
--pretend mode. I know I've noticed this myself and even heard
complaints about this behavior but surprisingly I can't find an open bug
to close.
This can be used as a run-time check later to ensure that kdesrc-build
finds compatible versions of its component modules (though that isn't
done yet).
This also required updating the test suite, which rather surprisingly
doesn't seem to need further modification (probably since the test-suite
includes kdesrc-build as a prerequisite, which sets up the @INC
modifications required).
This is probably the most dangerous change up to this point just because
a lot of modules descend from GitUpdate. I've run 'ack' on the source
tree just to make sure I didn't miss any renames from other modules at
this point. I've also built a kde-project module (which uses the new
ksb::Updater::Git indirectly) and it looks like everything still works
fine.
Michael Pyne
David Faure
Alex Merry