[kwlug-disc] So why not tar -cf tarball.tar a.xz b.xz c.xz, instead of tar -cJf tarball.tar.xz a b c ?

[Chris Irwin]

On Fri, Oct 21, 2016 at 06:09:44PM -0400, B.S. wrote:
>On 10/21/2016 05:31 PM, William Park via kwlug-disc wrote:
>>On Fri, Oct 21, 2016 at 02:05:37PM -0400, B.S. wrote:
>>>In any case, it isn't a reason / doesn't explain why tar'ring gzip's (to
>>>gain confidence of individual file integrity) isn't more prevalent.

It isn't prevalent because it isn't practical.

>>How would you improve
>>     tar -cf - /
>
>Did you really mean to tar all your filesystems to stdout, there?

That's the actually spot on, and why things are the way they are. We 
have cpio/tar/pax (if you swing that way) and compress/gzip/bzip2/xz.
Each group are separate classes of utilities that do very different 
things:

* Create an archive, storing metadata and data of many files.
    
* Compress a single data stream. It doesn't even know what a file is.

For example:

    $ tar -cf - / | gzip > root.tar.gz

This is simple. Make an archive, then compress it.

Ditto to extract:

    $ zcat root.tar.gz | tar -xf -

Now, this is common enough that some tar maintainers added support for 
compression directly within tar. At least GNU and BSD tars support it, 
though other implementations may not.

This gives us the following shortcuts for the above (which we're all 
familiar with):

    $ tar -czf root.tar.gz /
    $ tar -xzf root.tar.gz

But note that those are just shorcuts for convenience. The output file 
is identical to the piped commands earlier. This is great, because when 
a new compression format is becomes popular, I don't need to update tar 
to experiment with it. I can simply install the compression utility 
itself.

For example, GNU and BSD tar utilities added support for xz in 2009 and 
2012 respectively (from some basic historical manpage searches). This 
means RHEL5 didn't ship with it, nor did FreeBSD 8.2. But it doesnt 
matter, because xz was there, and you can just do:

    $ xzcat root.tar.gz | tar -xf -

Anyway, back to the original suggestion: Having tar compress the file 
data, rather than the archive. There is no single command line to do 
this. You can't pipe gz to tar. While you could modify tar to do this, 
there would you've now limited compatability with people running at 
least your updated version of tar.

Ultimately, you're not looking for tar. You're looking at a *new* 
utility that replaces tar + gzip. You might want to fork it from tar, 
and maybe even call it tar, but it won't be compatable with anybody 
else's tar.

It would also be an uphill battle to get it adopted, because there is 
very little benefit.

>>?
>
>The question is: With a damaged tar file, and the day -will- come, how 
>do you know which files within what remains are good and which ones 
>are damaged? (It's unlikely to be all of them.)

You don't. You throw the damaged file away, then you restore a good copy 
from a backup.

-- 
Chris Irwin

email:   chris at chrisirwin.ca
 xmpp:   chris at chrisirwin.ca
  web: https://chrisirwin.ca