This is mostly to make the testsuite run, which is not supposed to pull
down kde-build-metadata, though it may be worth it to construct a
suitable "fake metadata" instead.
The git code recognizes when a module that is selected for update/build
implicitly (e.g. because it is included as a subset of a module grouping
that *was* asked for) does not actually have a git branch. This often
happens for newly-introduced modules (which obviously won't already have
historical "stable" branches).
Rather than forcing users to know that a module is new and manually
ignore it we announce we will skip the build for these modules.
However I confused the return code in this function; no errors are
returned, only exceptions are used for errors, which meant kdesrc-build
would try to merrily build the non-existant module anyways.
Fixed by making this an error. I looked at making it possible to
completely elide the module from the update/build list entirely but that
requires a network call (we can't rely on cached source code since
we can't download non-existent sources), and I don't think that's
a good idea. The error message can probably be worded better though.
BUG:392654
FIXED-IN:18.04
_verifyRefPresent was also broken for tag-based checkouts. Since this
already uses determinePreferredCheckoutSource, which already used
refs/tags/ for tags, this code also doubled-up on tag names.
This whole code path would be a great candidate for that test suite
under t/ if I can ever get it polished up one day :(
To avoid commit spam, I've tested this on my own Qt module directories
by deleting them and forcing them to be checked-out with tags.
Turns out the original code was doubly-broken, since
determinePreferredCheckoutSource was already adding refs/tags/ to
returned git tags.
Rather than possibly breaking other users of that function by removing
it there, instead strip off refs/tags/ during clone if present.
git-clone doesn't like this value as an option to --branch/-b. Instead
just leave it as a plain tag name and hope it doesn't conflict with any
branch names.
Perl seems to have settled on the 'parent' module instead of manually defining
variables that tell the interpreter which packages to fall back to. Some
classes now use parent, some still use the old way.
Migrate everything over to be consistent.
Maintains a user-friendly improvement from a few years back to make it
easier to use module-sets to grab a wide set of modules without having
to disable individual modules that weren't present at that time.
This should be unnecessary for new clones (git does have an option to
not pull tags however). More importantly it's actually broken (we need
to cd into the newly cloned directory if we want to do it anyways).
JSON::PP is included in core Perl so one less dependency to look after,
and it's fast enough (for what we need) that there's really no point to
making it possible to look for JSON::XS.
This option can be used to setup the user.name and user.email git-config
options within each new git repository cloned or installed from
snapshot, including kde-projects repositories. This can be useful for
KDE developers with multiple Git identities that they use for
development.
Module settings override global settings so you could e.g. have a global
git-user option for KDE modules and then override specific modules or
module-sets with a different git-user setting.
This setting is only applied during the initial git-clone process
though... if there's interest in having kdesrc-build re-apply settings
later that's possible too.
BUG:352478
FIXED-IN:15.09
When the initrepo script fails the extracted snapshot directory (the
srcdir) is deleted, but it's also the current working directory, which
breaks git when doing the fallback clone. Fix by switching to an
existing directory.
CCMAIL:faure@kde.org
Perl core includes a "HTTP::Tiny" module that does what little we need,
and has had it since 5.14 or so, so prefer that instead and remove the
requirement to install libwww-perl.
This also required removing the one minor instance of the URI module,
which is not technically in Perl core but was installed due to LWP.
Instead we just use a regexp 'blessed' by the URI module author.
This is a 2-part change:
1. Always download needed metadata, even in --pretend mode. This means
that even --pretend won't stop kdesrc-build from trying to hit the
network, but --pretend was always intended to let you test command lines
to ensure you got the right build list before you wasted hours trying to
build something. If you don't want kdesrc-build to touch the network
ever, don't run it. :)
2. Check for JSON decoding errors specifically (since it may happen that
the JSON is corrupted on download somehow, or accidentally broken
upstream).
This should allow "kdesrc-build -p --print-modules", or even
"kdesrc-build -p" to work even as the very first command ever run.
BUG:340481
FIXED-IN:1.16
There seems to be nothing better than checking for the existence of
files or directories under .git at this point (though it's possible to
decode git-status --porcelain output without too much trouble in the
case of in-progress merges).
This should hopefully help avoid kdesrc-build trying to update a module
that is being worked on.
BUG:334324
FIXED-IN:1.16
Bug 341159 complains about kdesrc-build-setup not setting up the kde:
git prefix, but kdesrc-build-setup itself never even calls git, it just
creates a .kdesrc-buildrc and exits.
However the currently recommended way to start out kdesrc-build is to
run "kdesrc-build --metadata-only" for the first build to setup
directories and download required data, while still letting a subsequent
"kdesrc-build --pretend" do something useful. kdesrc-build didn't setup
the kde: prefix in time to support --metadata-only, and I'm willing to
bet that's what the user was encountering here.
So, setup the git configuration before the first time we can use git.
BUG:341159
FIXED-IN:1.16
Fixes bug 342806, where a kdesrc-build user notes that "Cloning"
messages are indented while "Updating" messages are not, which can be
confusing.
BUG:342806
FIXED-IN:1.16
We use a mix of error codes and raw exceptions in the git updater.
Fix this to use the exception objects used elsewhere (and which give
better error messages).
This also simplifies the code by removing error code returns from some
internal functions whose return value is then just converted into an
exception anyways. Easier just to throw the exception directly and
simplify the higher-level functions.
The one nuance here is that installing a git snapshot can fail yet that
should not cause an exception, we'd just fallback to doing a normal
git-clone.
The recent port to Getopt::Long broke the ability to use --foo=bar type
of options (where <foo> is a global option name known to the build
context). It also replaced the --module,foo=bar options with
--set-module-option-value=module,foo,bar.
While it's possible to use the latter syntax for global options (by
using 'global' as the module name) that is undocumented and may not work
forever. And either way, --foo options did use to work, and I noted it
was broken in the refactor.
So now that someone else has noticed I've reimplemented the feature.
It's difficult to do entirely from within Getopt::Long since the
documentation for that module gives you essentially one catch-all, which
only supports *non-options*.
So what I've done instead is to have kdesrc-build dynamically introspect
its list of global options and flags and then add them to the list of
valid options we pass to Getopt::Long. This also should mean that we
don't need to pass 'pass_through' as an option to Getopt::Long anymore,
though we'll see.
This code path won't be used for options that already have a command
line option (such as --async/--no-async) so that is still a misfeature
compared to how it was before, but I guess it can't be perfect. =D
I've tried to ensure that "flags" support the 'false' value to mean
boolean false, but be careful to pass false or 0 if that's what you
mean... specifying --dont-foo without giving it a value defaults it to
true.
BUG:330386
This is a long-overdue change that allows for overloading the
stringification operator to produce better error messages.
BuildException was previously not an actual module since kdesrc-build
itself was just a single script file. Now that we allow modules on disk
there is no reason to leave the exception class as a figment of Perl's
imagination.
Before this we would try to apply branch-group to a module, it would
fail (since plain git modules don't *have* any kde-projects metadata),
and we would invariably select master instead of choosing based on the
plain git module's branch option.
Note that this still doesn't solve the problem of modules that are from
kde-projects but *don't* have branch-group metadata; if both module
branch-group and global branch are active for such a module, the
module-specific branch-group will be tried and resolve to 'master'
independent of the global branch setting.
Whether this is a feature or not is not yet certain. :)
Now the priority order for all of the various "I want to checkout this
commit" options are centralized in one spot, and both cloning and
updating obey the same options now (they didn't before).
We need a better name for this feature.
But basically, this allows for creating kde-project repository-wide
"branch groups". For instance, you could list the git branch to use for
the "stable Qt/KDE 4" release/maintenance branch, the "new development
Qt/KDE 4" development branch (typically master, but that has been
changing), and the "upcoming next major release" branch (for us, this
means 'frameworks' sometimes, means 'master' sometimes, other modules
may have different ideas).
By using the 'branch-group' option (which remains to be further
documented) you can say that you want to defer to the metadata about the
repository branches stored in
kde-build-metadata/logical-module-structure. kdesrc-build should even
track branches for you as they are updated by the Release Team, if you
simply want to hang out on stable but compile your own stuff.
Note that this support requires Perl's JSON module (which should come
with Perl 5.14 if I understand it right, but you probably want to
install JSON::XS from CPAN). JSON support is only required for this
feature, it is not a new overall dependency.
See also: http://community.kde.org/Infrastructure/Project_Metadata
The previous version worked on modules that had *no* user (a setup most
of mine had as the user was set in the ssh config).
For modules with a user set the trailing '@' was included in the uid
which make the check fail even when the module was correct. (Though I
had thought I had tested this case with other modules).
In the past some KDE SVN modules used the "svn external" feature to pull
in other SVN repositories. These svn-external links were on the *server*
side so the link URL had to use a single scheme.
This scheme was https, which caused quite some problems for anonsvn
users just trying to update from SVN being an unattended kdesrc-build
run, as the SSL cert for svn.kde.org was not signed by a well-known CA.
As a result kdesrc-build updated svn automatically with the expected KDE
SVN SSL signature to avoid blocking on the interactive warning.
This is no longer required. The modules that used svn externals have
pretty much moved onto git now. The new svn server does not even have an
https interface, so this is dead code now, and removed accordingly.
Though this only works for Git modules, it should at least fix the
annoying "OMG YOUR REPO MOVED" warnings that never get fixed when in
async mode (which is the default).
Previously you simply never saw the warning due to the now-fixed output
squelching, now we need to fix this for good as well.
Michael Pyne
David Faure
Alex Merry