Implemented import + export for "Zip64 Extended Information Extra Field",
(in "Local file header" and "Central directory file header")
and for Data descriptor.
Focused only to be able to handle files with over 4GB uncompressed size,
in the zip archive.
The 64k filecount, and the 4GB compressed size limit is probably still present
Tried to follow pkware .ZIP File Format Specification,
Some cases were not clear to me and/or some zip compressing tool may not
perfectly follow the standard, like 'extra field' should be 28 bytes long,
but its reader now can read shorter (or longer) 'extra field'.
Replaced some 32bit codes with 64bit codes, in stream handling, in deflater.
Tested with an ods file that contained a content.xml that bigger then 4BG+
(import + export + reimport) on windows.
I think 4GB+ files import/export would be too slow fot unittest.
So, for unit test, used the small but zip64 format files,
that was attached to the bugzilla tickets
Note: It helps with Bug 128244 too (1 of the unittest tests it),
but that ods file missing manifest.xml, so LO won't be able to import it.
Change-Id: Idfeb90594388fd34ae719677f5d268ca9a484fb1
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/147306
Tested-by: Jenkins
Reviewed-by: Michael Meeks <michael.meeks@collabora.com>
And simplify the latter, to always use "CDATA" type (as the former did).
"CDATA" was used in all cases but one, where an empty string was used.
Change-Id: I1b3bfae40e29628e4094d9a6e58a69a66865874c
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/145526
Tested-by: Jenkins
Reviewed-by: Mike Kaganski <mike.kaganski@collabora.com>
to use in places where we know we can use the TempFileFast mechanism.
Speeds up export to EPUB from 43s to 17s on my windows machine.
This is essentially a copy of OTempFileService, but it does not
implement XTempFile (because these temp files have no name on Windows).
Also remove a couple of calls to Flush() (which is not necessary now),
and TellEnd(), which is a little slow on Windows.
Change-Id: Iced300bf8121eae08c7a011d62ec68b93330d84c
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/141598
Tested-by: Noel Grandin <noel.grandin@collabora.co.uk>
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
...so that its TOOLS_WARN_EXCEPTION can be used in
comphelper/source/misc/logging.cxx in a follow-up commit. (And while at it,
rename from diangose_ex.h to the more appropriate diagnose_ex.hxx. The
comphelper module is sufficiently low-level for this immediate use case, so use
that at least for now; o3tl might be even more suitable but doesn't have a
Library until now. Also, for the immediate use case it would have sufficed to
only break DbgGetCaughtException, exceptionToString, TOOLS_WARN_EXCEPTION,
TOOLS_WARN_EXCEPTION_IF, and TOOLS_INFO_EXCEPTION out of
include/tools/diagnose_ex.h into an additional new
include/comphelper/diagnose_ex.hxx, but its probably easier overall to just move
the complete include file as is.)
Change-Id: I9f3222d4ccf1a9ac29d7eb9ba1530d53e2affaee
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/138451
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
...so rename it to writeBytes for clarity, and drop the redundant return value.
Also clarify that it has a narrow interface and requires nBytesToWrite to be
non-negative.
Change-Id: I76dee83fecd6350f473f55dcffb950c16aa22d93
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/137169
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
so we can
(a) control the lifetime of the temp file properly
and
(b) we don't need to use file URLs which means we stay on the
efficient path on Windows
Change-Id: I2b1eed5f5afc8f89f53bde35ebb44e9922889817
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/136557
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
All call sites already effectively asked for an unsigned return type, including:
* The ThreadPool ctor took an nWorkers argument of type sal_Int32, but
internally stores that as std::size_t mnMaxWorkers.
* ZipOutputStream::reduceScheduledThreadTasksToGivenNumberOrLess apparently
benefits from an unsigned nThreadTasks parameter, getting rid of various casts
in its implementation that were necessary to silence signed vs. unsigned
comparison warnings.
The only drawback is that
comphelper::ThreadPool::getPreferredConcurrency() * 4
in package/source/zippackage/ZipPackageStream.cxx would now silently wrap around
instead of causing UB on overflow (which could be detected with appropriate
tools). Ideally, it would use some o3tl::saturating_mul if we had that, so add
a TODO comment for now.
While std::thread::hardware_concurrency returns unsigned, it looked more natural
to go with std::size_t here, as some call sites already used that (see above),
so the implementation of ThreadPool::getPreferredConcurrency is a natural place
to hide clamping std::thread::hardware_concurrency() to std::size_t (in the
unlikely case that std::size_t is of smaller rank than unsigned).
This required addition of o3tl::clamp_to_unsigned in o3tl/safeint.hxx.
Change-Id: I0a04a8b32e63ebfeb39f924c4b38520455a6fb38
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/135309
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
similarly to ByteReader
move both of them down to comphelper, since we want to use it from
comphelper, and comphelper is "below" unotools in the module dependency
graph
Change-Id: Ic98fa2268e125fd8e4378fb899ad5f97de721713
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/134645
Tested-by: Jenkins
Reviewed-by: Noel Grandin <noel.grandin@collabora.co.uk>
...where a signed and an unsigned value are compared, and the signed value has
just been proven to be non-negative here
Change-Id: I9665e6c2c4c5557f2d4cf1bb646f9fffc7bd7d30
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/133442
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
That these are only flagged when building against libc++ and not against
libstdc++ is a consequence of the plugin's dependence on implementation details
of the relevant classes, but so be it.
Change-Id: I6b96f81939edab2b8e618aa53c9fe23492edfbbd
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/131562
Tested-by: Jenkins
Reviewed-by: Stephan Bergmann <sbergman@redhat.com>
Caolán McNamara