BLOG: RSYNC

About RSYNC

If you have cygwin installed, rsync is a great tool to have. Basically rsync uses a transfer protocol that works on very small diffs. For example, say you have a huge 500M text file that changes frequently but in small increment (like a log file), you can use rsync to "synchronize" the remote file (usually the sourcE) with a local file. Rsync will then apply a hash to small blocks in the local and remote file and only exchange those hashes. When a hash does not match, the block that failed will then be downloaded and swapped into your local file.

The really cool thing about rsync is the ability to not only apply this concept to large files but also to large directory structures (say svn?)

So, to see a list of files available in the rsync repository, you would use:

rsync rsync://YourWorkStation/

This will actually show you a list of modules that are currently available.

rsync rsync://YourWorkStation/YourProject

will show you the list of directories under that modules.

To download the latest version of the SDK you would first have to figure out which version is available:

rsync rsync://YourWorkStation/YourProject/nightly/

drwxr-xr-x 4096 2006/11/20 09:25:04 . drwxr-xr-x 4096 2006/11/17 09:20:59 3.4.0.89 drwxr-xr-x 4096 2006/11/20 16:20:01 3.4.0.90 drwxr-xr-x 4096 2006/11/20 15:12:06 3.4.0.93

Here you can see that the latest would be 3.4.0.93

To download all the latest builds for all the platforms you can use:

rsync -PWa rsync://YourWorkStation/YourProject/nightly/3.4.0.93 .

This will :

Give you progress on the download (-P),

Download the whole file instead of the increment (-W),

Recursively archive the directory (-a)

Create a local directory 3.4.0.93 will all the file within.

Now if you just want one file for one platform you can then go:

rsync -PW rsync://YourWorkStation/YourProject/nightly/3.4.0.93/setup.exe .

About RSYNC and Subversion

Rsync has one really tricky bit when it comes to recusive download. It really important to understand the difference between downloading a directory and the content of a directory so the following:

rsync -PWa rsync://YourWorkStation/YourProject/nightly/3.4.0.93 .

is not the same as

rsync -PWa rsync://YourWorkStation/YourProject/nightly/3.4.0.93/ .

is not the same as

rsync -PWa rsync://YourWorkStation/YourProject/nightly/3.4.0.93/* .

if you do not put a trailing / at the end of the rsync url, rsync will create a directory. If you do, then it will not create the directory. If you put a star (*) then you will only get the files that are NOT hidden (not file that start with ".").

So to get the latest copy of SVN you can use the following:

rsync -Pa --del rsync://YourWorkStation/YourProject/svn/ .

This will:

Give you progress (-P), Recursively archive (-a), Delete any files that are not in the source (--del : this is a must to remove deprecated files and build artifacts). Will NOT create a directory SVN and dump all the files under that directory in the current directory.

Use svn info . to make sure you have the right version of the YourProject source.

Tony