Electronics with passion: amplifiers and power supplies, FET's and vacuum tubes..
linux

Web site backup and upload: rsync over sshfs (GoDaddy or other hosting with ssh access)

Other good people provided ideas for this solution:
notbrainsurgery @ livejournal
Robert @ softwarerevisions.blogspot.com

For some reason GoDaddy jailed hosting does not provide rsync, while it allows ssh user sessions to the host. It's not obvious how to move big chunks of data without "resume" functionality. Even worse for backup needs... Here's one of possible solutions based on my own experience. I hope this post could help someone too 🙂

Note that the "$" sign designates commands entered by the local user account, not root.
The "#" sign designates commands entered by root. Can also be done using sudo. Root access is required on the local host only.

  • Install the SSHFS package 

On Ubuntu it was not there by default. Thus I had to do:

$ sudo apt-get install sshfs

 

  • Setup open keys authentication from your local host to the provider

Note: ssh-keygen needs to be called only once on each host. In case you have the keys already set up - skip that one.

$ ssh-keygen
$ cat .ssh/id_rsa.pub

Copy the key to the clipboard

Login to the web-hosting machine:

$ ssh myuser@mysitename.xyz

do there:

$ ssh-keygen
$ vi .ssh/authorized_keys

Paste the key into the file and save the file:

i
Shift-INS
ESC
:wq
ENTER

Set the correct attributes of the file:

$ chmod 0600 .ssh/authorize_keys
  • Check that the key authentication works

The command should log you on to the hosting machine without requiring a password.

$ ssh myuser@mysitename.xyz

 

  • Create a mount point on local host
# mkdir /mnt/mysite
# chown localuser /mnt/mysite

 

  • Mount using SSHFS
$ sshfs myuser@mysitename.xyz: /mnt/mysite/
  • Set the group ID as at destination

Sorry, but right now I do not have enough motivation to dig why it's required 🙂
Rsync was throwing errors at me:
rsync: chgrp "/mnt/mysite/DestinationDir/a-schematic-diagram.pdf" failed: Permission denied (13)
I logged to the remote machine and checked the goup id:

$ id
uid=8123457(myuser) gid=100450(inetuser) groups=100450(inetuser)

Then on a local machine I did:

# cd /home/localuser/SourceDir/
# chgrp 100450 *

 

  • Transfer data

In my case I needed to upload tons of PDF files to the hosting machine.

$ rsync -avPO /home/localuser/SourceDir/* /mnt/mysite/DestinationDir/

 

-a, --archive         archive mode; equals -rlptgoD (no -H,-A,-X)
-v, --verbose         increase verbosity
-O, --omit-dir-times  omit directories from --times
-P                    same as --partial --progress

Some suggest using -z option too. Here I doubt we want to zip and unzip the data while transferring it from one local directory to another, even though one of them is mounted on a machine far away 😉

  • Unfortunately the connection in my case was dropping quite often.

Thanks to rsync re-connecting does not mean re-sending all data.

# umount -fv /mnt/mysite
$ sshfs myuser@mysitename.xyz: /mnt/mysite/
$ rsync -avPO /home/localuser/SourceDir/* /mnt/mysite/DestinationDir/

Could be wrapped in a script?...

I've seen what could be a better solution to the SSHFS hangs:

$ sshfs -o reconnect -C -o workaround=all myuser@mysitename.xyz: /mnt/mysite/

I am still testing this...

  • Remember to un-mount

Once everything is done:

# umount -fv /mnt/mysite

Please send me a link to your site, where this advise helped 😉

 

UPDATE 2012-03-20:

All the hassle described above can be avoided if one uses a smart SFTP client. FileZilla did all the magic for me.

4 Comments

  1. Greece

    Simply desire to say your article is as astounding. The clearness in your post is just great and that i can think you’re knowledgeable on this subject. Fine along with your permission let me to clutch your feed to keep up to date with approaching post. Thanks 1,000,000 and please keep up the gratifying work.

  2. Création site internet | Web Design | Programmation | Flyers [ Web Agence informatique

    I’ll immediately take hold of your rss feed as I can not to find your e-mail subscription link or e-newsletter service. Do you have any? Please allow me recognize so that I may subscribe. Thanks.

    • serge

      OMG! While moving from WP multisite I’ve forgotten to install e-mail subscription form. Thanks for the hint – fixing it in a moment.

  3. serge

    I am sorry to say that, but after more than half a year of struggle with GoDaddy I am about to move to another hosting. GoDaddy is simply way too slow. Especially when the America wakes up – the site becomes nearly inaccessible. They told me to check installed plugins. BS was that – a fresh install of WP worked also as if the hosting company used 8080 CPUs for their servers and FDDs for the storage. I am saying goodbye to GoDaddy ASAP. Money lost, time lost, visitors lost (the latter is hard to measure, but is unavoidable).

    Please accept my personal apologies – all our visitors who had to wait for this site to load way too long!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.