Using SSH to Move Things

It comes up every once in a while that you want to make a tarball of something but can't, because the machine hosting the files you want to tar doesn't have enough space. Or, you want to copy some big mess of files across the network, but want to transmit them compressed, to speed up the transfer, without having to first tar and compress them, then copy them, and then uncompress them, and then remember to delete the tarball on the source machine.

In either case - and in other cases - it's ideal to be able to accomplish the migration in swell foop (fell swoop). SSH comes to the rescue. The command:

ssh -l   tar c[j|z]O  | tar x[j|z]

will do the trick nicely.

It works because you can give SSH a command after specifying the host IP, and SSH will login, execute the command, and logout when it's done. Any output to STDOUT or STDERR will be routed to the machine making the call to SSH, so you can pipe or redirect that output just as you would had the remote command originated on the local machine.

So in the above command, we use ssh to log into machine ''ip'' with username ''name'' and ask tar to create an archive of ''path'', sending output to STDOUT. The "[j|z]" bit will allow you to simultaneously bzip2 (j) or gzip (z) the tarball, as it is created, before it appears on STDOUT. On the other side of the pipe, we ask tar to expand the input it reads from STDIN; if we have used "j" or "z", we need to give it to tar here, too.

This is particularly elegant if the account on the remote (source) machine has an authorized key for the account on the local (destination) machine. In that case, you don't have to enter a password or do anything other than issue the command.

You can move all sorts of things this way, not just tarballs. Any program that can write its output to STDOUT and read input from STDIN can be used this way. I've used a similar incantation to move Subversion repositories from one machine to another and even to block-by-block copy the low-level contents of one hard disk to another, across the network.