especiall when creating installsets with high parallelism, the packaging
stage can hang until faked-tcp is killed.
So avoid this race-condition altogether by switching to the sysv backend
if available.
Change-Id: If554abb8fb51218e6055e70c98622be77ecce58e
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/205896
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
Tested-by: Jenkins
…by a simple/static $(gb_CustomTarget_workdir)/foo
The build system has a lot of overly complicated leftovers from when it
was introduced and had not only deal with split repositories but also
had to coexist with another buildsystem. Along with lots of copy'n'paste
along the years the makefiles became hard to grasp for newcomers with
all our calls and evals.
As a first step to streamline that, the macros from TargetLocations that
simply prefix a static path to the argument (and similar of the same
kind) are a natural pick before simplifying the rules themselves/getting
rid of a bunch of eval statements.
Change-Id: Ia06dbbcd5d1994755a2ff05b84f72ccbc4e3cab5
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/167005
Tested-by: Jenkins
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
changing it to git-bash perl can probably be done, but would require
more changes to the path handling in the packaging code
Change-Id: I9a31ee6e9f122a2c167e11f5b4f73b18c5c0fa81
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/166343
Tested-by: Jenkins
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
it used the wrong variable name in AC_SUBST and also had no place where
it would be set for the rest of the build to use.
Also the script hardcodes the location of the WiX Toolkit, so check for
the same path in configure.
Also it was needlessly tied to LIBO_TEST_INSTALL - since it has its own
conditional, "double-guarding" it is not necessary.
Change-Id: I6dd4a41e63d2a43a3e2f1aac5b6799a6601eb656
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/159510
Tested-by: Jenkins
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
since it is a build-tool it needs to match the arch of the platform the
build is performed, already had used ILIB_FOR_BUILD, but CXX was missed
and broke the build for the aarch64 daily tinderbox
Change-Id: I79edf7d3f7813b43bb447d580d071914a5053b95
with high parallelism there's a high risk of running into random
failures when calling WiLangId.vbs via cscript.
The limiter doesn't use make's jobserver since it is too easy to
deadlock the build since all jobs are started at once, consuming all
slots, but in addition all wait for an additional slot that never is
made available because all jobs are blocked waiting....
All jobs being started at once and all jobs getting started from that
point on getting put under the limiter's control makes this simple
approach with separate grab/release calls possible. If they were spread
out the semaphore wouldn't be available (gets closed/removed as soon as
nothing waits for it anymore)
Change-Id: I345f2904a1d7e8989720722415fb51282ab3b05b
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/158886
Tested-by: Jenkins
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
6204336cc7 added support for building installsets
in parrallel using the GNU parallel tool, this patch changes that to use make's
own parallelism, dropping the need for the external tool.
By not creating huge commandline for all packages at once/sequentially, the
workarounds to modify the PATH variable can also be dropped.
Since : is a special character in makefiles separating a target from
prerequisites delimiter in the installer-tuples has been changed to the
hyphenation point.
The dependencies on the install.phony target have been moved to the actual
installset targets (and those are .PHONY targets as well since the target file
isn't actually generatead).
Also unify the tuples for the different platforms, even if the first element is
only used for Windows.
Move the msi templates to a separate directory for the different types, so that
packaging can also be done in parallel on Windows. Move the setup back to the
makefile so it can be setup prior to running the script.
Previoulsy the script used grep to remove comments from the msi template files,
and had to use "|| true" since grep would also return error when not finding a
match/on empty files. Switch that to awk so it can run without that workaround.
Change-Id: I2f8b73e04d0f601cb0b4e2d8352a38ef9957bc17
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/143679
Tested-by: Jenkins
Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
Seems on Windows, we're hitting the 8k character max cmd line length,
on setups with slightly deeper src tree path locations. Shorten length
of expanded call_installer calls by putting shell script into PATH
locally.
Change-Id: If1cddab9e4e07a7c5ebfae7a4e88e43b1bc8b907
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/134938
Tested-by: Jenkins
Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
If available, use GNU parallel to run N make_installer.pl scripts in
parallel, to scale packaging LibreOffice up with the rest of gbuild.
* fallback if no GNU parallel found - run make_installer sequentially
as before
* push most of the make_installer.pl input param tweaks from gbuild
down into a shared call_installer.sh script
* call gnu parallel with generated number of
"templ:lang:prodname:ext:pkgfmt:strip-flag" tuples, one for each
package to build (empty templ for non-windows, to save on cmd line
length)
* such that we can run all those in parallel (taking into account
the build's PARALLELISM parameter)
* there's still the main package build running epm sequentially for
umpteen sub-packages from within _one_ make_installer.pl instance, but
that's much harder to parallelize from inside Perl (so we punt on
that here)
Change-Id: Ie7d3084ed60d003d587c5e64dc9fb1809b23e409
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/133957
Tested-by: Jenkins
Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
So we can use pigz or other parallelizable tools if available. Shaves
off noticeable build time when packaging install sets.
- figure out if pigz is available (fallback to gzip otherwise)
- pass compression tool down into make_installer
- and handle as one of many global options there
Change-Id: Ia9d1ea27a9f990874238b6f0be3e1fd30a662ec6
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/128469
Tested-by: Jenkins
Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
Anyway files are stored without spaces, so we should use same name.
Additionally removed PRODUCTNAME_no_spaces variable, it's just a
redundant alias.
Change-Id: I5b88ac3b07d06969f7639a25603392c9d662d3f1
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/108405
Tested-by: Jenkins
Reviewed-by: Thorsten Behrens <thorsten.behrens@allotropia.de>
See instructions in solenv/gbuild/Trace.mk . This generates a file than
can be viewed e.g. in the Chromium tracing view.
Change-Id: I5f90647c58ca729375525b6daed2d4918adc8188
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/88754
Tested-by: Jenkins
Reviewed-by: Luboš Luňák <l.lunak@collabora.com>
as with private key on crypto-smartcard you'd have to enter your pin
over a hundred times while creating full-lang builds and that is not
fun.
This reduces it to
* once for dll/exe
(at least in case for mergelib is is less than 350 objects and
that doesn't break commandline limits - previously it was set to
only sign 20 objects at a time, forcing a pin-entry over 15 times) and
* once for main installation set
* once for SDK
* once for all the helppacks
(signing description previously also contained the language, this
change drops that to just "<productname> <version> Helppack"
and last three are not scattered timewise, but are done after all
packaging is complete, so the build only waits twice for user-input.
Change-Id: Ibb8bb233e967556f9654573ad30d0ed5883b533f
Reviewed-on: https://gerrit.libreoffice.org/78649
Tested-by: Jenkins
Reviewed-by: Christian Lohmaier <lohmaier+LibreOffice@googlemail.com>
So "grep -v" returns != 0, if your input or output file is empty.
which breaks the Jenkins build, because Jenkins runs scripts with
-xe per default.
The "strip comments" grep currently fails for the empty file
instsetoo_native/inc_openoffice/windows/msi_templates/components.txt
Change-Id: I53a477702dbcc3c00c4a23263c3b402cb936250a
... after 39b04ad7cb started using
PRODUCTNAME but here it was LibreOffice_Dev for mysterious reasons
whereas PRODUCTNAME is LibreOfficeDev so rename for consistency.
Change-Id: Ia5f5a24f405923a7ea1641de65709fbf842df79c
i.e. "don't add gnomevfs requires to epm created rpms if GNOMEVFS is disabled"
seeing as setup_native/source/packinfo/packinfo_office.txt references
find-requires-gnome.sh and doesn't have conditional support revert the split
and put the the conditional into find-requires-gnome.sh
Change-Id: I34bd37c4fd252f2175ff055024293dea946b4881
since there are rpm-frontends that do their own version comparison that
is different from rpm itself.
Change-Id: Iddf38c14e7f48eec5d043de57e3404fcd9da24f7
debian packages don't cope with release number of 0, so use release 1
for debian master/dev-packages
Change-Id: Id91926322d13bddad3a39faccfee4e131d8956b0
It is constant and can just be replaced by $(SRCDIR)/solenv.
Use BUILD_TYPE where it was used to check if config_*.mk is sourced.
Change-Id: Ib9d480c57194b6340093aa47776f8768df69b7d1
...as per #libreoffice-dev IRC:
Sep 19 10:32:24 <mst__> sberg, moggi why the hell is that thing named
"cppunit/cppunittester" and inside a subdir? it's obstructing my attempt to
put it in $(INSTDIR)/program
Sep 19 10:33:28 <mst__> (... and if you wonder "wtf does it have to do with
INSTDIR" you have never heard of awesome LibreOffice_Test installset.... not
that i would know who needs it :)
Sep 19 10:36:36 <sberg> mst__, it is in a subdir of solver/*/bin so that on
Windows it would not accidentally have picked DLLs next to itself instead of
the module-local DLLs it was supposed to test (back when we had module-local
output trees)
Sep 19 10:37:02 <mst__> sberg, ahh hysteric reasons then, /me renames it
Sep 19 10:37:55 <tml> mst__, if nobody you know uses LibreOffice_Test, just kill
it?
Sep 19 10:38:59 <sberg> mst__, tml, LibreOffice_Test was conceived by pmladek
and/or kendy, IIRC
Sep 19 10:40:31 * kendy does not remember anything about it :-)
Sep 19 10:42:17 <sberg> wasn't that something so users (or QA people?) could
easily run the smoketest against an installation, to see whether the
installation is any good at all, by installing that LibreOffice_Test alongside
the installation proper?
Sep 19 10:43:26 <sberg> mst__, ...and I'd unscientifically vote to kill it
Sep 19 11:34:23 <pmladek> mst__, sberg: I have created the LibreOffice_Test
package for one QA guy. He does not longer work on LO. I am not sure if anyone
else started to use it. So, I think that it can be killed.
Oct 17 18:18:07 <tml_> sberg: have you ever noticed that when you try to
actually run instdir/unxmacxi/LibreOfficeDev.app , the system actually tries
to run cppunittester inside the app bundle (it says so in the crash report)
(it crashes because cppunittester requires a specialized DYLIB_LIBRARY_PATH
apparently)
Oct 17 18:19:29 <tml_> I suspect that the system when cppunittester as part of
the build process is run from inside instdir (i.e. inside an app bundle) the
system "caches" this false knowledge, and thinks that the executable of the
app bundle is cppunittester...
Oct 17 18:19:36 <sberg> tml_, no, never noticed; with "run
instdir/unxmacxi/LibreOfficeDev.app" you mean calling "open
instdir/unxmacxi/LibreOfficeDev.app"? (I always call
.app/Contenst/MacOS/program explicitly)
Oct 17 18:19:52 <tml_> yes, I mean "open instdir/..."
Oct 17 18:20:53 <tml_> some googling tells me that at least years ago, the
CFBundleExecutable key in the Info.plist is ignored if it is manually changed,
so I guess similar caching of mapping between an app bundle and which
executable to actually run happens in this case
Oct 17 18:23:17 <tml_> and last year somebody even claims "And while on Mountain
Lion, CFBundleExecutable seems to be a no-op", which would be odd, surely
there must be widely used apps that have several executables inside the MacOS
directory; how would the system know which one to run when the app is run?
Oct 17 18:24:38 <tml_> hmm, apparently the code that handles this might be open
source even, http://www.opensource.apple.com/source/CF/CF-744.18/CFBundle.c
Oct 17 18:25:52 <tml_> some mention of "caches" there yes, my guesses might be
right
Oct 17 18:27:05 <tml_> if I cp -R instdir/unxmacxi/LibreOffice.app foo.app and
open foo.app, it works fine
Oct 17 18:28:33 <tml_> anyway, I guess it would be cleaner to have cppunittester
somewhere else even without this problem
Oct 17 18:37:09 <sberg> tml_, yes, IIRC having cppunittester in instdir was a
misguided mst decision, because that odd LibreOffice_Test product (that
pmladek said nobody needs any longer anyway) includes it; I think consensus
was to kill LibreOffice_Test and move cppunittester where all the other NONE
executables are, but looks like nobody executed
Oct 17 18:37:55 <tml_> ah ok, so mst should know what needs to be done? good, no
need for me to try to hack this now then
Oct 17 18:38:19 <sberg> tml_, I'll do the cleanup tomorrow, unless somebody
beats me
This removes smoketest/losmoketest et al along with the *_Test product, as they
seem to not make sense without it anyway. smoketest/Executable_libtest.mk
appears to be a test that could also be run during the build, and only ended up
in the *_Test product by accident, so I left it untouched for now.
Change-Id: I8024472c909fe0a885eb08ef4d3777f8a9e1f7c8
Just check for $(VERBOSE) or $(verbose) being non-empty instead of
checking for $(VERBOSE) equalling "TRUE". Isn't our normal way to do a
verbose make to pass verbose=t?
Change-Id: Ic4ddc1fe025fed55ca56fd21b615640c389c0454
We produce standalone -langpack packages elsewhere. This should get the
size of the main dmg for MacOS X back to normal.
Change-Id: Ifd0cb1f9e31fe211c15636ddb6653e3672570819
Christian Lohmaier