2009
09.17
When I was uploading a large file from server A to server B, the server B’s disk space ended up, so scp broke transfer and printed an error message.
SCP sent more than 4GB data (90%), so I was looking for a solution on how to resume the previous file transfer. Unfortunately, scp can’t resume file transfer, but … rsync can! In rsync manual we find –partial options:
–partial
By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances it is more desirable to keep partially transferred files. Using the –partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster.
Now we can write a command like this:
rsync --partial --progress -e 'ssh' file_to_upload my.server-b.com:path
2009
09.17
Today our customer told us about a problem with one of the his website. Sometimes he couldn’t upload a file (PowerPoint presentation). The problem is strange, because in logs we didn’t find an error message.
His website use ruby on rails on the apache (passenger) webserver and nginx for http proxy server. In the beginning I thought it was a problem with rails application, but customer’s programmer checked the code three times and did’t find any problem with an application.
After an hour of searching I looked in nginx logs, where I found this:
2009/09/15 14:59:08 [error] 94387#0: *65836224 client intended to send too large body: 1475115 bytes, client: XXX.XXX.XXX.XXX, server: *.example.com, request: "POST /admin/attachments.js HTTP/1.1", host: "www.example.com", referrer: "http://www.example.com/foo/bar"
The first thought that came to my mind was “Hmm to large filesize”? I went to nginx wiki looking for options that set max upload filesize. I found an option named client_max_body_size. This option sets the max body of upload filesize. Default value is 1 Mb (1M). I changed this options to 5M, so the problem disappeared :)