Chris Musty's picture

Hi again guys,

I have been actively testing tklbam in the past few weeks and have found many ways it will help and many ways it is limited by infrastructure - not tklbam's fault, none the less it can't be used in certain circumstances.

So after a weekend of uploading and getting to 13Gb I decided it was not (or at least the uplink was not fast enough) for my uses and cancelled by hitting Ctrl Z. Now under my Hub account I see I have 2 backups that are stuck with "First backup" where one failed from a bad download and the other I stopped.

Is this a bad thing to do and are there any implications anywhere for doing this? Also is there a better way to stop a backup if necessary (I cant see why, aside from speed issues, but who knows).

Forum: 
Alon Swartz's picture

Firstly, ctrl+z will "stop" (read as suspend) a process, it won't "stop" the process as you might presume. To stop execution of a process you should use ctrl+c.

$ cat test
#!/bin/sh
while true; do echo -n "."; sleep 1; done

$ ./test
...^Z
[1]+  Stopped                 ./test

$ jobs
[1]+  Stopped                 ./test

$ fg 1
./test
....^C

Regarding upload speed, depending on the latency of S3 you might want to try launching an instance in Amazon EC2 in the region closest to you and using rsync to transfer your data across. Then on the instance use tklbam to encrypt and backup the data to S3.

I haven't tested this myself, but it might be faster for a large bulk of data - not sure. Amazon also provide a service where you can send them a harddrive and they dump the contents to S3.

Lastly, if you have backups in the hub stuck in the "first backup" state, please use the feedback option in the Hub and include the backup id's you want us to delete, and we'll get that done (I have on my todo list to add this functionality to the Hub's UI, so that should make it easier for users in a similar position).

Chris Musty's picture

Its upload bandwidth not S3 that is the problem. Uploads here are 125Kb/s (Kilo Bytes) at best.

Doing what you suggest might speed things up but it kinda defeats the purpose of using TKLBAM - simplicity.

I think I am going to use Proxmox to backup the image and then look at placing it on another filesystem and TKLBAM-backup from there or possibly use the alternate "--address" feature with a second file system kind of like a cache system I guess. The problem will be that the local network is 1000 times faster and can swamp the temp server before it can get an image into S3.

Has anyone else solved this issue in a particular way?

One day when we get fiber and connections arn't so asymetrical (or at least better than now) I will be happy.

Just how much faster would EC2 be that S3?

Chris Musty

Director

Specialised Technologies

Add new comment