The logfiles generated for each module are stored in a log directory (again for
each module). The converse is also supposed to apply, a log directory should
only exist for a module if it has logs.
kdesrc-build enforced this in the past by taking extreme care not to call
"getLogDir" unless needed. At some point I missed that nuance and used that
call elsewhere, even when no log was to be generated.
This had the effect of creating a new log directory for the module that had its
log directory looked up, but without ever adding logs to it. kdesrc-build would
also move the latest symlink to point to this empty directory, and then later
remove the directory containing the actual set of latest logs during cleanup.
Even worse, this would happen for every module (since we forced a call to
"getLogDir" so that later code in new async processes would all resolve to the
same log directory)
I fix this by splitting up the "getLogDir" functionality into a read-only part
(creates the base log directory but not the module log directory and does not
move latest symlinks), and a read-write part for log_command.
The message gets a bit more annoying as the number of warnings increases. I
store the number of warnings as well so that future versions can give a really
prominent message if new warnings are introduced since the last build.
Right now this only makes some of the code intent a bit clearer (saying "was
successful" instead of checking against non-bool values). But this would also
be useful for tracking stats on compile warnings.
This fixes bug 395627, where the error is actually that we set
CMAKE_PREFIX_PATH twice if qtdir is set to a non-system path and the
user is also setting CMAKE_PREFIX_PATH.
Unfortunately the second value overrides the first (the one the user
set).
Also added a test for this, which fails before the fix and passes
afterwards. The full test suite (all 5...) pass.
BUG:395627
FIXED-IN:18.08
This is mostly to make the testsuite run, which is not supposed to pull
down kde-build-metadata, though it may be worth it to construct a
suitable "fake metadata" instead.
This adds a testcase that would fail without the bugfix for bug 394497
and now passes.
This did require undoing some of the changes to generateModuleList but
the only real behavior change (beside merging duplicate ignored module
checks) was to move the step that generates module dependencies to occur
earlier so that it can happen before the filter steps
CCBUG:394497
A test case will follow but would clutter the actual fix (which is
ensuring that a pass to remove all ignored modules happens immediately
prior to forming the final module list).
BUG:394497
FIXED-IN:18.08
Prior to this change, CMAKE_PREFIX_PATH was consistently set (which is
preferred for most find_package() calls) but CMAKE_MODULE_PATH was only
set for Qt5 (if Qt had a different prefix than the module being
installed).
This could lead to situations where "simplified" find_package() calls
would use "Module" mode (using CMAKE_MODULE_PATH) and end up falling
back to a search of system directories. This could result in finding the
wrong module if the module being searched for was installed on the
system.
Summary:
Ignore /usr as prefix or kdedir when prepending environment variables to not
overrule user setting of PKG_CONFIG_PATH.
For me it was enough to fix my build with prepending CMAKE_PREFIX_PATH
conditionally, but in theory the other environment variables should also only
be prepended if it is not /usr to prepend.
Test Plan:
Compiled XCB 1.13 to /opt/xcb and have XCB 1.11 in /usr/.... Set
PKG_CONFIG_PATH to /opt/xcb/lib/pkgconfig.
Without this patch CMake, called by kdesrc-build, finds XCB 1.11 in
/usr/lib/x86_64-linux-gnu according to its log, but sets as version number
1.13 for some reason.
On make this fails. It still uses the lib from /opt/xcb> but does not include
the correct headers in opt/xcb/include.
With the patch CMake directly finds the lib in /opt/xcb and build succeeds.
Reviewers: bshah, mpyne
Reviewed By: mpyne
Subscribers:
Tags:
Differential Revision: https://phabricator.kde.org/D12739
The Mojolicious work exposed this bug but it's a bug all the same. The
way that Perl's glob function works differs based on whether you use it
in a scalar (including boolean) context or whether you use it in a list
context.
I had the parentheses in this test to try to force a list context but
that's apparently not sufficient as I had every fourth Qt module
failing to build due to this glob failing, in my Mojolicious testing.
The revised test successfully forces glob() to use list context, which
means it always returns the same values for a given directory instead of
acting as a non-reentrant iterator.
The git code recognizes when a module that is selected for update/build
implicitly (e.g. because it is included as a subset of a module grouping
that *was* asked for) does not actually have a git branch. This often
happens for newly-introduced modules (which obviously won't already have
historical "stable" branches).
Rather than forcing users to know that a module is new and manually
ignore it we announce we will skip the build for these modules.
However I confused the return code in this function; no errors are
returned, only exceptions are used for errors, which meant kdesrc-build
would try to merrily build the non-existant module anyways.
Fixed by making this an error. I looked at making it possible to
completely elide the module from the update/build list entirely but that
requires a network call (we can't rely on cached source code since
we can't download non-existent sources), and I don't think that's
a good idea. The error message can probably be worded better though.
BUG:392654
FIXED-IN:18.04
Summary: The qtdir config option from .kdesrc-buildrc was seemingly innefective, which is fixed by this patch
Reviewers: mpyne
Differential Revision: https://phabricator.kde.org/D11736
_verifyRefPresent was also broken for tag-based checkouts. Since this
already uses determinePreferredCheckoutSource, which already used
refs/tags/ for tags, this code also doubled-up on tag names.
This whole code path would be a great candidate for that test suite
under t/ if I can ever get it polished up one day :(
To avoid commit spam, I've tested this on my own Qt module directories
by deleting them and forcing them to be checked-out with tags.
Turns out the original code was doubly-broken, since
determinePreferredCheckoutSource was already adding refs/tags/ to
returned git tags.
Rather than possibly breaking other users of that function by removing
it there, instead strip off refs/tags/ during clone if present.
git-clone doesn't like this value as an option to --branch/-b. Instead
just leave it as a plain tag name and hope it doesn't conflict with any
branch names.
There is code to avoid using the TTY code that updates the status of the
build in the current line. This code prevents using the fancy TTY stuff
if progress output is not supported, but doesn't actually ever output
the end result of the build (success or failure), which wasn't the
intent.
In the other situations we don't want to clutter the line output with a
redundant status. Instead of duplicating the "progress output
supported" check later to output the status (or not), I just took out
the check for whether progress output is supported entirely, since the
progress callback can work even with build systems w/out support.
BUG:391396
FIXED-IN:18.04
The warning here complains about conditions which are already pretty
much fixed from outside of the kdesrc-build user population anyways.
It's still something notable so instead of removing completely I
downgrade it to a debugging message.
BUG:388657
FIXED-IN:18.02
The move to JSON storage of persistent opts break first-time users,
since the refactored code never actually gets around to setting a
persistent_options key in the build context object.
This seemed to work previously with Data::Dumper, but JSON::PP is not
happy with being asked to serialize an undef. On the other hand I had
always thought that sub setPersistentOption would have 'autovivified'
the needed hash table, and the reporter's build process should have
resulted in many calls to setPersistentOption over the build.
BUG:388180
FIXED-IN:18.01
My attempt to make the KDEProjectReader's check for YAML be a runtime check was
thwarted because I had separately used a "use KDEProjectReader" elsewhere,
which Perl ensures happens at compile time.
Instead make the check part of the constructor so that the package can at least
compile without throwing an error.
Perl seems to have settled on the 'parent' module instead of manually defining
variables that tell the interpreter which packages to fall back to. Some
classes now use parent, some still use the old way.
Migrate everything over to be consistent.
YAML is not an appropriate fallback from what I can glean of Perl
community discussions. YAML::XS is preferred, or YAML::Syck. If I can
get repo-metadata to conform to a stricter YAML subset then YAML::Tiny
would work too.
CCBUG:388109
This is needed to try to perform filtering of the module list based on
which git branches are available, which we could do easily when using
kde_projects.xml.
Without that data, we effectively have to wait until the module list is
already made and about to git update a module before we can tell if the
branch will be there. By that time it's too late to have filtered the
module list, so we have to determine whether this should be a hard error
or not later in execution.
To make that choice, we essentially need to know whether the module is
being built because it was specifically asked for or whether it was
brought in as part of a larger module set (and therefore can be ignored
if the git branch is missing).
This adds tracing for the config file to supplement existing tracing of
cmdline selectors.
Maintains a user-friendly improvement from a few years back to make it
easier to use module-sets to grab a wide set of modules without having
to disable individual modules that weren't present at that time.
Qt5 modules built from git rely on git information being available to
generate private headers and the like, so don't strip out the .git
directly when using builddir hack.
Likewise, using the "-r" flag to force a full recursive search for .pro
files causes qmake to error out on some modules where the .pro isn't
meant to be built as part of the module build (e.g. examples).
Many work (e.g. qtxmlpatterns) work just fine with qmake using srcdir !=
builddir, but most do not and instead assume a srcdir == builddir.
This is difficult for kdesrc-build (we generally assume we can delete
everything under the build directory) so reintroduce the old builddir
hack that symlinks all the files in the srcdir under the builddir.
To avoid needlessly throwing symlinks around for the existing
QMake-using modules (which apparently all already work), this only
applies to modules with a repository under qt.io.
Note that qtbase should be handled by a separate build system (still
called "Qt4.pm" and doesn't fall under this hack.
The log directories are stored in a central directory in the
BuildContext so that we know which ones we're supposed to reuse. But
this only happens on first use. If this first use happens during an
async build it won't be properly persisted, so each succeeding build
will think it's the first one to need a log directory.
This can happen for modules that use a log directory different from the
global directory.
This "last-build-rev" persistent option is probably more appropriate for
Module.pm anyways, since there is already a similar "last-install-rev"
check in there.
Yuri Chornoivan
Michael Pyne
Roman Gilg
Luiz Romário Santana Rios