Though this only works for Git modules, it should at least fix the
annoying "OMG YOUR REPO MOVED" warnings that never get fixed when in
async mode (which is the default).
Previously you simply never saw the warning due to the now-fixed output
squelching, now we need to fix this for good as well.
In synchronous mode it was impossible to get the build subroutine to
attempt to build a module which had no source update, as the updater
code would filter out its build phase *in addition* to marking it as
up-to-date.
This never broke the async code as filtering out the build phase was
always unimplemented (which is what led to a specific IPC type for "code
up-to-date" in the first place).
The build thread owns the TTY in the current setup. To keep the update
and build processess from fighting over the TTY when in async mode,
we've been simply squelching the update output up to this point, except
for errors.
This is not really the best idea as it can leave out kind of important
details. With this change we configure the logging subsystem to forward
its messages that it would ordinarily print out to instead go over an
IPC channel.
The IPC channel knows which module is currently being operated on, tags
the message, and forwards it over to destination proc's IPC handler,
which extracts it and holds onto the message until that proc reaches the
same module.
Once the receiver proc is processing the module, those messages are all
output at once and then normal processing resumes.
In case some messages are never read out this way (e.g. a module is
updated but never built) we have a message scavenger right after the
build is done to give those messages as well.
It's kind of annoying to be unable to build all modules in a certain
kde-projects module grouping, *except* for some certain chosen ones.
Now you can: You can add the ignore-modules option in your module-sets
using kde-projects repository.
It accepts a space-separated list of module names (either kdefoo or
kde/module/kdefoo style) to ignore.
Although the intention is to only filter out matching modules in *that*
module-set, it would actually perform the filtering even if you
accidentally put it in the wrong module set due to a fluke of the
implementation (I recommend not relying on this, however).
The documentation has also been updated, including an example of the
usage.
For whatever dumb reason I coded the "ignored kde-projects modules"
handling (kde-build-metadata/build-script-ignore) by having the code
that reads in the list of ignored modules go and immediately update the
build context.
For obvious reasons this only works when the update is actually
performed, which means ignore information wasn't used in pretend mode.
We fix this by moving the build context update to the rest of the
metadata handling.
This commit removes the 'megaclass' code that IPC::Pipe has become to
support two different dataflow paths in a single class.
Instead, we use up to 2 different IPC objects in async mode to give the
dataflow needed.
Splitting this up required moving the message boundary detection code
back to IPC::Pipe. It is now expected that a given IPC subclass is able
to pass messages with message boundaries intact, and to block while
waiting for messages if concurrency is supported. IPC::Null already
maintains message boundaries and doesn't support concurrency.
Previously IPC and IPC::Pipe cooperated to perform message boundary
detection, by inserting and removing '\n' characters. This was always
kind of a hack. Now we simply pass the message length across the pipe
followed by the message, and ignore the idea completely in IPC.
The impetus for all of this had been to improve performance, but
unfortunately that hasn't happened here, although it's still a positive
change from the code simplicity front.
Specifically, when we have a non-master global branch, the new feature
to filter out module-set modules that don't have that branch was being
mis-applied to kde-build-metadata just to get some flags set right.
Instead, set the flags needed directly and skip the filter.
No need to re-add /usr if it's already in PATH (and doing so can even
lead to changing a user's preferred PATH order, even though it's
possible to work around that issue by playing with user_path in
.config/kde-env-user.sh).
BUG:312122
FIXED-IN:1.16
The most noticeable effect is probably that it should be damn near
impossible not to have automoc build first now, at least in comparison
to other kde-projects modules. It also should take care of the vast
majority of the code needed for bug 312324. Still needed is handling for
Qt/Qt5.
BUG:312324
FIXED-IN:1.16
We now read in and should be able to handle branch-specific dependencies
and module dependency exclusions.
For full support we still need to find a way to properly handle Qt vs.
Qt5 modules (e.g. for phonon Five and automoc).
The last commit's first attempt had an error where the kde/foo/* syntax
would be stripped to '*', which the KDEXMLReader actually understands
perfectly. Unfortunately a bare '*' reads in all modules, including
websites and other unpleasant things.
I only caught this by chance, so add a sanity check to make this an
error in case I miss it in the future.
Although a module like kde/kdeutils/kcalc will eventually end up with a
$module->name() of "kcalc", the module starts off with whatever the user
typed in for its name in the use-modules declaration.
This makes it difficult to override its options in a later module / end
module declaration (as its documented to do). E.g. the kcalc branch for
the following should be "KDE/4.9".
module-set
repository kde-projects
use-modules kde/kdeutils/kcalc
branch KDE/4.10
end module-set
module kcalc
branch KDE/4.9
end module
*** Important ***: There's no way for kdesrc-build to know when parsing
that the "module foo" is supposed to override a module-set module if it
hasn't seen the foo declared yet. In this example, if you simply did
"use-modules kdeutils" then you may end up with a svn-based kcalc (since
it's a "new" module without a repository declaration).
I'm not sure right now if I'll decide that possibility needs to be
handled as well so I'll leave the bug open. But it should be possible to
override any module-set module that is named with this fix.
CCBUG:299415
Commit 63e21e9e1b filtered only based on a
module-set branch. This causes problems with global branches that might
have the same issue so we'll just filter out all such modules.
We probably need to use the scm()'s getBranch() method to have a 1:1
check in the presence of use-stable-kde, but even that should work due
to the way branch handling is performed with use-stable-kde (which
should always pick existing branches).
BUG:307694
This should fix the big ugly error messages for this particular
scenario. The code in question is trying to determine a good branch name
that it can use, which means it need to verify that the requested branch
name *doesn't* exist. This would correspond to an exit code of 1 for the
given git command.
In pretend mode all safe_system() calls give an exit code of 0, so it
appears that there are no safe unused branch names.
Since even in pretend mode the source directory should exist in this
code path I switched to use a system() call (with the correct full
branch name to boot).
I'm not sure the other part of the bug is fixed (i.e. checking out a
local branch immediately after cloning or whatever dfaure was seeing) so
I'll leave open for now until I can do further testing.
CCBUG:311758
Or in other words, command-line modules that are "guessed" kde-project
modules (and eventually confirmed to be kde-project modules) do not have
their options loaded from the rc file as they would if no modules were
passed on the command line. This could be highly annoying when
considering things like branch switches.
It's rather inefficiently fixed for now by brute-force "proj-expanding"
all modules available in the configuration file since that's the easiest
way to propagate module-set options into a list of modules.
Unfortunately there's massive low-hanging fruit for optimization here.
BUG:312096
FIXED-IN:1.16
In the process of trying to plan out the best way to fix the issue where
module names passed on the command line that are implicit subprojects of
a kde-projects module go unrecognized, I decided that I needed a better
way to navigate the API.
There is a Doxygen::Filter::Perl available which seems decent as it
allows one to use the nice and familiar Doxygen tool, but it seems a bit
buggy still.
NaturalDocs is a tool which seems better able to cope with kdesrc-build
and its code, although it prefers comments to be formatted unlike the
normal qdoc/JavaDoc syntax used by Doxygen. However it's not that much
different and is easier to install and use otherwise. Maybe when I get
some time I'll go ahead and properly convert everything.
Michael Pyne
Yuri Chornoivan
Luigi Toscano
Script Kiddy
Daniel Laidig
Montel Laurent