I imagine such binaries would be picked up by anti-virus but it would be cool to be able to compress data with the latest algorithms without needing the client to have the matching decompression libraries. Unfortunately it was a no-go to ship decompression binaries with embedded data, but I thought it would be a cool idea to use cosmopolitan libc to create a platform agnostic binary which when run uses gzip/bzip/lz4 to quickly decompress the data in a platform agnostic way. I calculated that with the space savings of gzip we could ship a gzip binary embedded with the data in a smaller size than the deflate data on its own.
UNRAR WINDOWS 10 ARCHIVE
I could compress a +100GB data archive in under a minute using pigz, but since we had to support windows it would take over 30 minutes to compress the file using deflate. It truly is amazing how terrible compression support has been in windows. I begrudgingly cursed windows and reverted my change, robbed of my small victory. It turns out that there were still windows 10 versions out in the wild that don’t support bzip2 zip files.
Fast forward a month or two later I get word that one of our clients is having problems uncompressing the data we sent them. Auto-unrar qBittorrent Windows 10, qBittorrent v3.2.3 Upon completion, to automatically extract. I checked the zip standard and saw that bzip2 compression was added in 2001 so clients have had 20 years to implement it.Īfter changing our data delivery pipeline to bzip2 I patted myself on the back for finding an easy win which significantly reduced data size and increased compression speed. A few years ago I was working on optimizing scientific data delivery by switching our compression algorithms to use bzip2 instead of deflate64 for our zip files.
0 kommentar(er)
