The logfiles generated for each module are stored in a log directory (again for
each module). The converse is also supposed to apply, a log directory should
only exist for a module if it has logs.
kdesrc-build enforced this in the past by taking extreme care not to call
"getLogDir" unless needed. At some point I missed that nuance and used that
call elsewhere, even when no log was to be generated.
This had the effect of creating a new log directory for the module that had its
log directory looked up, but without ever adding logs to it. kdesrc-build would
also move the latest symlink to point to this empty directory, and then later
remove the directory containing the actual set of latest logs during cleanup.
Even worse, this would happen for every module (since we forced a call to
"getLogDir" so that later code in new async processes would all resolve to the
same log directory)
I fix this by splitting up the "getLogDir" functionality into a read-only part
(creates the base log directory but not the module log directory and does not
move latest symlinks), and a read-write part for log_command.
Summary:
Ignore /usr as prefix or kdedir when prepending environment variables to not
overrule user setting of PKG_CONFIG_PATH.
For me it was enough to fix my build with prepending CMAKE_PREFIX_PATH
conditionally, but in theory the other environment variables should also only
be prepended if it is not /usr to prepend.
Test Plan:
Compiled XCB 1.13 to /opt/xcb and have XCB 1.11 in /usr/.... Set
PKG_CONFIG_PATH to /opt/xcb/lib/pkgconfig.
Without this patch CMake, called by kdesrc-build, finds XCB 1.11 in
/usr/lib/x86_64-linux-gnu according to its log, but sets as version number
1.13 for some reason.
On make this fails. It still uses the lib from /opt/xcb> but does not include
the correct headers in opt/xcb/include.
With the patch CMake directly finds the lib in /opt/xcb and build succeeds.
Reviewers: bshah, mpyne
Reviewed By: mpyne
Subscribers:
Tags:
Differential Revision: https://phabricator.kde.org/D12739
The Mojolicious work exposed this bug but it's a bug all the same. The
way that Perl's glob function works differs based on whether you use it
in a scalar (including boolean) context or whether you use it in a list
context.
I had the parentheses in this test to try to force a list context but
that's apparently not sufficient as I had every fourth Qt module
failing to build due to this glob failing, in my Mojolicious testing.
The revised test successfully forces glob() to use list context, which
means it always returns the same values for a given directory instead of
acting as a non-reentrant iterator.
This "last-build-rev" persistent option is probably more appropriate for
Module.pm anyways, since there is already a similar "last-install-rev"
check in there.
The qtdir setting took on much less importance with Qt 5. We've never
officially supported building Qt 5 with kdesrc-build, and qmake and
cmake both use qmake or similar Qt-provided config information to find
Qt, rather than using the unsupported QTDIR environment variable.
As a result, default the value to an empty string (interpreted as use of
system Qt) and stop adding it automatically to the environment variables
defined during the build. If set, it will continue to be used for now.
This also helps the environment setup driver detect when it should look
for the system Qt5 qmake.
Basically, modules which are updated by default but not actually built
(e.g. due to using the manual-build option) are never mentioned in the
kdesrc-build output in the default async mode of operation.
This required a bit of refactoring of the IPC code to fix, since the
underlying cause is that the output during async mode is driven by the
needs of the build thread, which would request updates from the monitor
thread on a per-module basis. But the build thread never inquired about
modules only being updated, so any such modules were never mentioned on
the command line.
This is to allow for reading in "options" blocks that may apply to
module-sets later, even if the needed module-set hasn't been read-in yet
(e.g. if it's contained in a common file meant to be included in some
but not all configurations).
To do this right we need to decouple the actions of reading options and
then later applying them to the various OptionsBase subclasses. This is
a start toward that.
Now we describe up-to-date module progress (if multiple modules are
being built) and consolidate muliple lines of information into just a
couple (for build and install).
I noticed that I was getting error messages from a git usage after
install on my KF5 modules, which are configured to use
remove-after-install.
Turns out that git or the shell or kernel or something doesn't like
running a command from a now-invalid cwd.
Otherwise you get an interesting interaction with --refresh-build and
remove-after-install options: remove-after-install removes the previous
build directory, and --refresh-build tries to prune under the new
build directory, which fails since the Perl module that does this first
tries to detect the cwd and fails (since we removed it).
Adds ignore-kde-structure option which will fetch and compile all
projects in the same dir, meaning that instead of:
extragear/network/kde-telepathy/ktp-text-ui
it would be just
ktp-text-ui
By moving this code and associated utility methods into Application we
nearly complete the long-in-progress refactoring of the monolithic
kdesrc-build script into proper classes/objects.
The kdesrc-build script itself is nearly bare, but there's still a bit
more to do.
At least it should be easier to make individual test scripts to test
different bugs instead of having a monstrosity of a kdesrc-build-test
script that has to maintain a common state.
I've tested this in each way that I'm able (run needing to make a new
source dir, build dir, new install and log dir, run updating existing
checkouts, bzr/svn/git/kde-projects all tested, etc.). But this was a
large move so let me know if I've missed anything.
This is a long-overdue change that allows for overloading the
stringification operator to produce better error messages.
BuildException was previously not an actual module since kdesrc-build
itself was just a single script file. Now that we allow modules on disk
there is no reason to leave the exception class as a figment of Perl's
imagination.
We need a better name for this feature.
But basically, this allows for creating kde-project repository-wide
"branch groups". For instance, you could list the git branch to use for
the "stable Qt/KDE 4" release/maintenance branch, the "new development
Qt/KDE 4" development branch (typically master, but that has been
changing), and the "upcoming next major release" branch (for us, this
means 'frameworks' sometimes, means 'master' sometimes, other modules
may have different ideas).
By using the 'branch-group' option (which remains to be further
documented) you can say that you want to defer to the metadata about the
repository branches stored in
kde-build-metadata/logical-module-structure. kdesrc-build should even
track branches for you as they are updated by the Release Team, if you
simply want to hang out on stable but compile your own stuff.
Note that this support requires Perl's JSON module (which should come
with Perl 5.14 if I understand it right, but you probably want to
install JSON::XS from CPAN). JSON support is only required for this
feature, it is not a new overall dependency.
See also: http://community.kde.org/Infrastructure/Project_Metadata
This is unfortunately a giant change, as all of the functionality that
is encompassed into module-sets currently had to migrate over to
multiple separate classes, including the new ksb::ModuleSet class and
subclasses.
This was a long-overdue change, however, and should allow for accurately
tracking a source module-set for a given module.
On the other hand this migration of logic has made it easier to
understand each of the individual pieces where they stand (e.g. there is
no longer a separate expandXMLModules and expandModuleSets).
In addition we can properly handle ignore-modules with wildcards just as
we do with use-modules (they even use the same matching logic) which
means that it is safe to integrate this into master (assuming no extra
boogs get added, of course).
This will also help with fixing some of the extant module-selection bugs
(321883, 299415).
BUG:321275
CCBUG:321667
Once I figure out a "preferred" solution for this use case I'll probably
convert to that, until then no reason to pollute with warnings.
I've run a search for other uses of given/when and didn't see any.
Though this only works for Git modules, it should at least fix the
annoying "OMG YOUR REPO MOVED" warnings that never get fixed when in
async mode (which is the default).
Previously you simply never saw the warning due to the now-fixed output
squelching, now we need to fix this for good as well.
In synchronous mode it was impossible to get the build subroutine to
attempt to build a module which had no source update, as the updater
code would filter out its build phase *in addition* to marking it as
up-to-date.
This never broke the async code as filtering out the build phase was
always unimplemented (which is what led to a specific IPC type for "code
up-to-date" in the first place).
In the process of trying to plan out the best way to fix the issue where
module names passed on the command line that are implicit subprojects of
a kde-projects module go unrecognized, I decided that I needed a better
way to navigate the API.
There is a Doxygen::Filter::Perl available which seems decent as it
allows one to use the nice and familiar Doxygen tool, but it seems a bit
buggy still.
NaturalDocs is a tool which seems better able to cope with kdesrc-build
and its code, although it prefers comments to be formatted unlike the
normal qdoc/JavaDoc syntax used by Doxygen. However it's not that much
different and is easier to install and use otherwise. Maybe when I get
some time I'll go ahead and properly convert everything.
This can be used as a run-time check later to ensure that kdesrc-build
finds compatible versions of its component modules (though that isn't
done yet).
Michael Pyne
Roman Gilg
David Faure
Jeremy Whiting
Àlex Fiestas