Compressing: gzip, bzip2 and xz

Making a file smaller

A compression program rewrites a file to take up less space, and can restore it exactly. gzip is the classic one. It replaces the file with a .gz version:

ls -l logs/day-01.log
gzip logs/day-01.log
ls -l logs/day-01.log.gz

In the terminal (the original is replaced by the smaller .gz)

student@vulntech:~$ ls -l logs/day-01.log
-rw-r--r-- 1 student student 17892 Oct 17 15:05 logs/day-01.log
student@vulntech:~$ gzip logs/day-01.log
student@vulntech:~$ ls -l logs/day-01.log.gz
-rw-r--r-- 1 student student 1073 Oct 17 15:05 logs/day-01.log.gz

gzip -d (or gunzip) puts it back:

gzip -d logs/day-01.log.gz
ls -l logs/day-01.log

In the terminal

student@vulntech:~$ gzip -d logs/day-01.log.gz
student@vulntech:~$ ls -l logs/day-01.log
-rw-r--r-- 1 student student 17892 Oct 17 15:08 logs/day-01.log

Three tools, a trade-off

There are three common compressors. They trade speed against how small they squeeze things: gzip is fast, xz squeezes hardest but is slowest, bzip2 is in between. They all take -d to decompress:

cp logs/day-02.log a.log; cp logs/day-02.log b.log; cp logs/day-02.log c.log
gzip a.log
bzip2 b.log
xz c.log
ls -l a.log.gz b.log.bz2 c.log.xz

In the terminal

student@vulntech:~$ cp logs/day-02.log a.log; cp logs/day-02.log b.log; cp logs/day-02.log c.log
student@vulntech:~$ gzip a.log
student@vulntech:~$ bzip2 b.log
student@vulntech:~$ xz c.log
student@vulntech:~$ ls -l a.log.gz b.log.bz2 c.log.xz
-rw-r--r-- 1 student student 1068 Oct 17 15:06 a.log.gz
-rw-r--r-- 1 student student  588 Oct 17 15:06 b.log.bz2
-rw-r--r-- 1 student student  468 Oct 17 15:06 c.log.xz

Reading without unpacking: zcat

zcat prints a .gz file's contents without decompressing it on disk, so you can pipe it straight into grep or less. bzcat and xzcat do the same for the others:

gzip logs/day-03.log
zcat logs/day-03.log.gz | head -n 2
zcat logs/day-03.log.gz | wc -l

In the terminal

student@vulntech:~$ gzip logs/day-03.log
student@vulntech:~$ zcat logs/day-03.log.gz | head -n 2
day-03 entry 1: service running, status ok
day-03 entry 2: service running, status ok
student@vulntech:~$ zcat logs/day-03.log.gz | wc -l
400

The useful combination: .tar.gz

tar bundles but doesn't shrink; gzip shrinks one file. Together they make the archive you see everywhere, the compressed tarball. tar can do both in one step: add z for gzip (j for bzip2, J for xz):

tar -czvf logs.tar.gz logs
ls -l logs.tar.gz

In the terminal

student@vulntech:~$ tar -czvf logs.tar.gz logs
logs/
logs/day-01.log
logs/day-02.log
logs/day-03.log
logs/day-04.log
logs/day-05.log
student@vulntech:~$ ls -l logs.tar.gz
-rw-r--r-- 1 student student 5754 Oct 17 15:07 logs.tar.gz

Unpack it the same way, with -xzf. Modern tar even notices the compression by itself, but it's good to be clear:

mkdir out; tar -xzvf logs.tar.gz -C out
ls out/logs

In the terminal

student@vulntech:~$ mkdir out; tar -xzvf logs.tar.gz -C out
logs/
logs/day-01.log
logs/day-02.log
logs/day-03.log
logs/day-04.log
logs/day-05.log
student@vulntech:~$ ls out/logs
day-01.log  day-02.log  day-03.log  day-04.log  day-05.log

Remember: gzip/bzip2/xz shrink a file; -d restores it. zcat reads without unpacking. tar -czf x.tar.gz folder makes a compressed archive, tar -xzf x.tar.gz unpacks it.