I have a lot of tar and disk image backups, as well as raw photos, that I want to squeeze onto a hard drive for long term offline archival, but I want to make the most of the drive’s capacity so I want to compress them at the highest ratio supported by standard tools. I’ve zeroed out the free space in my disk images so I can save the entire image while only having it take up as much space as there are actual files on them, and raw images in my experience can have their size reduced by a third or even half with max compression (and I would assume it’s lossless since file level compression can regenerate the original file in its entirety?)
I’ve heard horror stories of compressed files being made completely unextractable by a single corrupted bit but I don’t know how much a risk that still is in 2025, though since I plan to leave the hard drive unplugged for long periods, I want the best chance of recovery if something does go wrong.
I also want the files to be extractable with just the Linux/Unix standard binutils since this is my disaster recovery plan and I want to be able to work with it through a Linux live image without installing any extra packages when my server dies, hence I’m only looking at gz, xz, or bz2.
So out of the three, which is generally considered more stable and corruption resistant when the compression ratio is turned all the way up? Do any of them have the ability to recover from a bit flip or at the very least detect with certainty whether the data is corrupted or not when extracting? Additionally, should I be generating separate checksum files for the original data or do the compressed formats include checksumming themselves?
Honestly, given that they should be purely compressing data, I would suppose that none of the formats you mentioned has ECC recovery nor builtin checksums (but I might be very mistaken on this). I think I only saw this within WinRAR, but also try other GUI tools like 7zip and check its features for anything that looks like what you need, if the formats support ECC then surely 7zip will offer you this option.
I just wanted to point out, no matter what someone else might say, if you were to split your data onto multiple compressed files, the chances of a bit rotting deleting your entire library are much lower, i.e. try to make it so that only small chunks of your data is lost in case something catastrophic happens.
However, if one of your filesystem-relevant bits rot, you may be in for a much longer recovery session.
AFAIK none of those formats include any mechanism for error correction. You’d likely need to use a separate program like zfec to generate the extra parity data. Bzip2 and Zstandard are somewhat resistant to errors since they encode in blocks, but in the event of bit rot the entire affected block may still be unrecoverable.
Alternatively, if you’re especially concerned with robustness then it may be more advisable to simply maintain multiple copies across different drives or even to create an off-site backup. Parity bits are helpful but they won’t do you much good if your hard drive crashes or your house catches fire.
Lzip
Compression formats are just as susceptible to bitrot as any other file. The filesystem is where you want to start if you’re discussing archival purposes. All of the modern filesystems will support error correction, so using BTRFS or ZFS with proper configuration is what you’re looking for to prevent files from getting corrupted.
That being said, if you store something on a medium and then don’t use said medium (lock it in a safe or whatever), then the chances you’ll end up with corrupted files approaches 0%. Bitrot and general file corruption happens as the bits on a disk are shifted around, so by not using that disk, the likelihood this will happen is nearly 0.
Bitrot happens even when sitting around. Magnetic domains flip. SSD cells leak electrons.
Reading and rewriting with an ECC system is the only way to prevent bit rot. It’s particularly critical for SSDs.
deleted by creator
Upgrade from compression tools to backup tools. Look into using restic (a tool with dedup, compression and checksumming) on a filesystem which also checksums and compresses (btrfs/zfs) - that’s probably most reasonable protection and space saving available. Between restic’s checks and the filesystem you will know when a bit flips and that’s when you replace the hardware (restoring from one of your other backups).
Honestly amazing question, I’ve lost entire 7z archives because of minor amounts of bit rot
Error correction and compression are usually at odds.Error correction usually relies on redudant data to identify what was corrupted it also helps if the process for error correction is ran more frequent. So storing it away offline is counter to the correction and the added redundancy will reduce the space gains. You can check different error correction software or technique. Ex RAID. I recommend following the 3-2-1 data backup rule. Also even if you can’t do all the steps doing the ones you can, helps.Sidenote optionally investigate which storage brand/medium/grade you want. Some are more resistant than other for long term vs short term. Also even unused storage will degrade over time whether the physical components, the magnetic charge weakening or electric charge representing your data. So again offline all the time isn’t the best; run it a couple times a year if not more to ensure errors don’t accumulate.
Sadly I won’t give specifics because I haven’t tried your use case and I am not familiar, but hopefully the keywords help.
Error correction and compression are usually at odds.
Not really. If your data compresses well, you can compress it by easily 60, 70%, then add Reed-Solomon forward error correction blocks at like 20% redundancy, and you’d still be up overall.
There are a lot of smart answers in here, but personally I wouldn’t risk it by using a compressed archive. Disk space is cheap.
This is a money game, how much are you willing to invest?




