[PYTHON] Fast file transfer with fabric

Postscript: The method of http://qiita.com/NPoi/items/46364461f0ab76e986c3 is smarter!

Isn't fabric put slow? Let's use rsync. In the case of transferring the entire directory with many static files such as images, HTML, JS, etc., the execution time of fabric is dramatically different, so I will introduce it. (I am assuming people who use fabric to some extent)

Rsync as a non-root user

This is an official feature provided by fabric so it's easy to use.

from fabric.contrib.project import rsync_project

@task
def hoge():
	rsync_project(
		local_dir='./mydir',
		remote_dir='/usr/local/destination/mydir',
		exclude=['.DS_Tore', '*.tmp'],
		delete=True
	)

rsync as root user

After rsyncing as a non-root user, you can do mv or cp -r, but the nice thing about rsync is that you can only transfer the differences, so I created this function. The arguments are aligned with rsync_project.

The behavior is to rsync once to / tmp / (directory name hash value) / and then sudo rsync on the server.

It depends on the is_dir function of fabtools, so you need pip install fabtools

import hashlib
from fabric.api import local, run, sudo
from fabric.contrib.project import rsync_project
from fabtools.files import is_dir

def root_rsync(local_dir, remote_dir, exclude=[], delete=False):
	def _end_with_slash(dir_path):
    	if dir_path[-1] == '/':
    	    return dir_path
    	else:
    	    return dir_path + '/'
    local_dir = _end_with_slash(local_dir)
    remote_dir = _end_with_slash(remote_dir)
    m = hashlib.md5()
    m.update(remote_dir)
    me = local('whoami', capture=True)
    remote_tmp_dir = '/tmp/%s/%s/' % (me, m.hexdigest())
    run('mkdir -p %s' % remote_tmp_dir)
    if is_dir(remote_dir):
        run('rsync -a %s %s' % (remote_dir, remote_tmp_dir))  # already exists
    rsync_project(
        remote_dir=remote_tmp_dir,
        local_dir=local_dir,
        exclude=exclude,
        delete=delete
    )
    sudo('rsync -a %s %s' % (remote_tmp_dir, remote_dir))