The logfiles generated for each module are stored in a log directory (again for
each module). The converse is also supposed to apply, a log directory should
only exist for a module if it has logs.
kdesrc-build enforced this in the past by taking extreme care not to call
"getLogDir" unless needed. At some point I missed that nuance and used that
call elsewhere, even when no log was to be generated.
This had the effect of creating a new log directory for the module that had its
log directory looked up, but without ever adding logs to it. kdesrc-build would
also move the latest symlink to point to this empty directory, and then later
remove the directory containing the actual set of latest logs during cleanup.
Even worse, this would happen for every module (since we forced a call to
"getLogDir" so that later code in new async processes would all resolve to the
same log directory)
I fix this by splitting up the "getLogDir" functionality into a read-only part
(creates the base log directory but not the module log directory and does not
move latest symlinks), and a read-write part for log_command.
This is mostly to make the testsuite run, which is not supposed to pull
down kde-build-metadata, though it may be worth it to construct a
suitable "fake metadata" instead.
The chomp doesn't return the chomp'ed value (rather '1' to reflect
number of characters removed), and fixing that revealed another bug with
my recent change to this code where I forgot to `tee' debug output to
the tty.
Should be a bit faster, but major goal here was to decouple the progress
output from the details of what we're outputting to. Functionality is
still the same (except that ninja progress output now works too) but now
it's that much easier to permit additional means of progress updates in
the future (e.g. a GUI, web server, even a Plasmoid).
Perl core includes a "HTTP::Tiny" module that does what little we need,
and has had it since 5.14 or so, so prefer that instead and remove the
requirement to install libwww-perl.
This also required removing the one minor instance of the URI module,
which is not technically in Perl core but was installed due to LWP.
Instead we just use a regexp 'blessed' by the URI module author.
My last commit checked for missing executables before trying to execute
them, but still left open the possibility that it would be impossible to
call exec(2) on an executable that had just been found (for whatever
reason).
Closer inspection in 'perldoc perlipc', and 'perldoc -f open' has helped
illustrate how to catch that condition. E.g. it turns out that calling
'close' on a pipe-opened filehandle will automatically call waitpid,
plus I'd forgotten to exit in that error case in the child proc. (This
is why it would take 2-3 seconds after failing for the parent process to
continue).
The more interesting question is what error policy to use in this case.
Errors are expected in pretend mode (this function actually runs
processes even during pretend mode since otherwise it can be impossible
to determine whether to clone or update, switch branches, etc.), but on
the other hand during execution errors should be unexpected and
noteworthy.
FIXED-IN:15.04
BUG:345425
Perl's "open '-|' $program" feature, which is used to pipe the output of
a process to a filehandle variable, is implemented internally using
pipes along with a fork/exec construct. The pipes are created, your
process automatically forks, and the child exec(2)'s the command you
wished to run.
This is all documented, and I thought I had a handle on this, but Perl's
open() function's error checking is based only on whether fork(2)
succeeded, not on whether exec(2) succeeded.
What I think we get when we don't check for errors is some kind of
raised signal (e.g. SIGPIPE) which is then duly received by the existing
async-IPC code (which is already doing multi-proc), which then kills the
IPC procs due to the 'impossible SIGPIPE' happening and crashes
kdesrc-build.
This fix, to check for an executable before trying to run, should at
least make this kind of error more apparent (this isn't perfect yet
either since 'bzr' failing to exist during pretend mode should be listed
as an update failure, not update success). I would still need to find a
better way to handle this since this is still a race condition.
CCBUG:345425
I was noticing that sometimes Ctrl-C at the TTY doing a kdesrc-build -p
left a monitor process hanging at 100% CPU, not to mention all the
kdesrc-build processes spamming goodbye messages.
It didn't help that I was doing waitpid on an updater PID when that
updater PID had to be 0 by definition.
Either way, I've verified that the forking going on here shouldn't start
a different process group for the shell session, so ^C is already going
to all of the relevant processes. We just need to make sure to catch the
signal and do the right thing (i.e. exit the child process).
By moving this code and associated utility methods into Application we
nearly complete the long-in-progress refactoring of the monolithic
kdesrc-build script into proper classes/objects.
The kdesrc-build script itself is nearly bare, but there's still a bit
more to do.
At least it should be easier to make individual test scripts to test
different bugs instead of having a monstrosity of a kdesrc-build-test
script that has to maintain a common state.
I've tested this in each way that I'm able (run needing to make a new
source dir, build dir, new install and log dir, run updating existing
checkouts, bzr/svn/git/kde-projects all tested, etc.). But this was a
large move so let me know if I've missed anything.
This is a long-overdue change that allows for overloading the
stringification operator to produce better error messages.
BuildException was previously not an actual module since kdesrc-build
itself was just a single script file. Now that we allow modules on disk
there is no reason to leave the exception class as a figment of Perl's
imagination.
This can be used as a run-time check later to ensure that kdesrc-build
finds compatible versions of its component modules (though that isn't
done yet).
Moving ksb::Util required splitting out script version tracking to a
separate module, as ksb::Util was using the previously-global versionNum
variable automatically.
Yuri Chornoivan
Michael Pyne