File Compression via Command-line

This topic was published by and viewed 2494 times since "". The last page revision was "".

Viewing 1 post (of 1 total)
  • Author

  • DevynCJohnson
    • Topics - 437
    • @devyncjohnson

    Many file compression and archive formats exist, each one with its advantages and disadvantages. GNU/Linux and most Unix systems support most compression and archive formats. It is helpful to know the commands used to manipulate the different formats. This article will at least show the basic/general method of performing a compression and extraction, plus various other tricks.


    Archives are files that hold other files together as one file and lack compression (like Tar).

    ar - ARchiver (ar) is the predecessor to tar. Debian packages (*.deb) still use ar.

    • Archive: ar rcs NAME_ARCHIVE_FILE FILES
    • Extract: ar -x ARCHIVE.a

    CPIO - "CoPy In and Out" was once used on magnetic tape drives. CPIO is used today in for simple backups (other uses exist). The tar utility has largely replaced CPIO. The creation of CPIO files is more complicated than other archive formats. By that, I mean, the command is lengthier than other commands. As you read on, you will see that using CPIO is not like other commands.

    This may be a little off topic, but RPM files can be converted to CPIO files using the "rpm2cpio" command like this – "rpm2cpio FILE.rpm". Not all systems have rpm2cpio.

    • Archive: FILE-LIST | cpio -o > ./FILE.cpio
    • "FILE-LIST" may be the echo command with the file paths or a "find" command that will output a list of files.
    • Extract: cpio -id < FILE.cpio
    • List Contained Files: cpio -it < FILE.cpio

    Shell Archive (shar) - A shar file is a self-extracting archive that only depends on the sh utility. This file is a plain-text shell script that recreates the files on execution. This format is not secure. If the user desires an archive format to hide the contained files, do not use shar files. Obviously, since this is self-extracting, the file needs to be executed like any other application/script.

    • Archive: shar FILES

    ISO - An ISO file is a optical disc image or a file that mimics an optical-disc (think of virtual machines).

    • Archive: dd if=/dev/cdrom of=/tmp/cdimg1.iso
    • "if=/dev/cdrom" reads the optical disc and "of=/tmp/cdimg1.iso" is where the data is written. The ISO file will contain the disc's data including the filesystem.
    • Extract:
      mkdir /mnt/iso #make a directory
      mount -o loop FILE.iso /mnt/iso #Mount your iso file (FILE.iso) to the new directory
      cp * ~/ISO/ #copy all files in the ISO to a folder in $HOME

    tar - The tar archive is the most popular archive format for Unix and Unix-like systems. A tarball is a tar file. Most tarballs are compressed with some other compression format (the new file is still called a tarball). The reason for this is most compression formats are unable to compress multiple files together as one file. Also, Tar cannot compress files, so using Tar with a compression format solves such issues. Tarballs have various file extensions as seen below.

    • Short (Long) – Compression
      • .taz (.tar.z) - zip
      • .tbz, .tb2, .tbz2 (.tar.bz2) - bzip2
      • .tgz (.tar.gz) - gzip
      • .tlz (.tar.lzma, .tar.lz) - lzma
      • .txz (.tar.xz) - xz
    • Archive: tar -cf NEW_ARCHIVE.tar FILE
    • Extract:
      tar -xf ARCHIVE.tar #regular tar</li>
      	<li>tar -xfz ARCHIVE.tar.gz #compressed -> uncompress -> untar
    • List Contents: tar -tvf ARCHIVE.tar

    Compression Files

    Files that compress data so the resulting file consumes less storage (bytes).

    bz2 – Bzip2 offers more efficient compression than zip or gzip. Bzip2 can only compress one file. That means multiple files compresses with bzip2 will produce individual bzip2 files. However, Tar is used to archive the files into one (the *.tar file) and Bzip2 compresses this file. That is why a tar extension is on most bzip2 files. The "bzcat" command can be used on Bzip2 files the same way "cat" is used on regular files.

    • Compress: bzip2 -z NEW_ARCHIVE.bz2 FILE
    • Extract: bzip2 -d FILE.bz2 #bunzip2 = "bzip2 -d"

    Freeze (F) – Freeze is a compression format that can only compress one file (just like Bzip2). To view the contents of a file (like a text file) compressed as Freeze, use the "fcat" command - "fcat FILE.f | less"

    • Compress: freeze -x FILE #the new file will be FILE.f
    • NOTE: The "-x" makes the compression better at the cost of speed. This parameter can be removed if desired.
    • Extract: freeze -d FILE.F #melt = unfreeze = "freeze -d"

    gz – GNU Zip (Gzip) is a popular compression format that is faster but less efficient than Bzip2. Like Bzip2, Gzip can only compress one file. This is why Tar is often used with Gzip. Gzip was made to replace the "compress" utility/command in earlier Unix systems. To view the contents of files inside of Gzip files like using "cat" on other files, use the "gzcat" command. Gzip is the most popular compression utility in Gnu/Linux, so it is highly recommended that user familiarize themselves with Gzip's commands.

    • Compress: gzip -z NEW_ARCHIVE.gz FILE
    • Extract: gunzip FILE.gz

    lzma - The Lempel–Ziv–Markov chain algorithm (LZMA) compresses data better than Bzip2 and Gzip, but is slower than those two algorithms. Like Bzip2 and Gzip, LZMA can only compress one file which is why Tars are used when compressing many files (the *.tar counts as one file). The command "lzcat" acts like "cat" by showing users the contents of the compressed files.

    • Compress: xz --format=lzma NEW_ARCHIVE.lzma FILE
    • Extract: unlzma FILE.lzma
    • TIP: Make an alias for lzma to make it easier to use alias lzma='xz --format=lzma '. With this alias, users can type "lzma" instead of xz --format=lzma.
    • NOTE: Using the different compression cat-like commands (such as gzcat and others) even works on tar-containing files (like *.tar.lzma).

    xz - XZ uses the LZMA2 algorithm thus making it better than LZMA. Again, like other compression formats, to compress multiple files, use Tar and then XZ. The "xzcat" command is a cat-like command that allows users to view the contents of the compressed files.

    • Compress: xz NEW_ARCHIVE.xz FILE
    • Extract: unxz FILE.xz

    Compression + Storage Files

    Storage files with compression store and compress files. These formats do not require tar when compressing multiple files into one file.

    7z – The 7z compression format was first used in an application named "7-Zip", hence the filetype's name. 7z can compress multiple files together without the need of tar. 7z does not store file permissions, unlike other compression formats.

    • Compress: 7za a NEW_ARCHIVE.7z FILES
    • Extract: 7za e FILE.7z

    apk - Android installation packages are apk files. Android uses *.apk files just as Debian uses *.deb and Fedora uses *.rpm. APK files are created in the Android developer's IDE.

    arc – Arc does not require multiple file be merged together (tar) to be compressed.

    • Compress: arc NEW_ARCHIVE.arc FILES
    • Extract: arc x FILE.arc

    jar - Java classes (compiled Java-code files) are held together using a Jar file. Jar is not intended to be used as storage. It is meant to be an application file containing executable code. Jar files can also be used as addons/plugins for various applications.

    • Compress: jar cf NEW_ARCHIVE.jar FILES
    • Extract: jar xf FILE.jar

    rar – The Roshal Archive (RAR) is a commonly seen format, especially among Windows users. RAR is not open-source, but it is free. RAR supports password protection.

    • Compress: rar a NEW_ARCHIVE.rar FILES
    • Extract: rar x FILE.rar

    xar - eXtensible ARchive format is open-source archiving format. RPM5 uses xar.

    • Compress: xar -cf FILE.xar FILES
    • Extract: xar -xf FILE.xar

    zip - ZIP is a widely used compression format. ZIP supports password protected archives.

    • Compress: zip ARCHIVE_NAME FILES
    • Extract: unzip ARCHIVE_NAME

    zpaq - ZPAQ is a special type of compressor. When a newer version of a file is added to a ZPAQ file that contains the older version, ZPAQ performs an incremental update. This means the older version of the file exists in the archive.

    • Compress: zpaq ARCHIVE_NAME FILES

    zz - Zzip is a self-extracting archive. This means the file is "executed" and it will uncompress itself.

    • Compress: zzip ARCHIVE_NAME FILES

    Linux Scripting/Command-line Article Index -

Viewing 1 post (of 1 total)