This script imitates common shell commands available on Unix systems to access Amazon Storage Service(S3). I started to work on this script to integrate amazon storage service with existing backup scripts both at work and home. It also turned out to be a good way to make any open source code written by me available to everyone. In fact this readme file and the associated script will be available on S3 and were uploadedusing this script.

This code is still under development and does not handle a lot of errors. It will work fine as long as it does not encounter corner cases. I will keep improving at over time. Any comments, suggestions and patches are always welcome. I have a TODO section at the bottom which reflects my thoughts on future improvements.

• Changed script's name to s3cli.rb
• Added commands "printlog", "enablelog" and "disablelog"
• Fixed "get" and "put" to preserve file names during uploads and recreate the directory path during downloads if needed
• 
  

This script requires ruby and ruby gems to be installed. It also requires the "aws/s3" gem available from http://amazon.rubyforge.org. You will need to have an account with Amazon Web Services and an active Storage Service account. You should then lookup your keys and name from the account and put them in the amazon.keys file.

You will need to create a file called amazon.keys with the following format and edit the path to it in s3cli.rb for ruby to find it.

Contents of amazon.keys file should look like

$keys = {
        'secret' => 'xxxxxxx',
        'id' => 'yyyyyyyyy',
        'name' => 'username'
}

<id> means that user needs to give a name of his choice. For example a sample command for ls will look like "s3cli.rb ls designandcode"

[ ] represents an optional argument.

| separates all the possible values for an argument. Only one of the possible values can be specified at a time.

• s3cli.rb ls [<bucket>]
• s3cli.rb ls_l [<bucket>]
• s3cli.rb mkdir <bucket> [public|publicrw|private]
• s3cli.rb put <file1> [... <filen>] <bucket> [public|publicrw|private]
• s3cli.rb get <bucket> [<file>]
• s3cli.rb rm <bucket> <file>
• s3cli.rb rmdir <bucket>
• s3cli.rb rmdir_f <bucket>
• s3cli.rb geturl [https] <bucket> [<file>] <num<m|h|d|w|y>>
• s3cli.rb printlog <bucket>
• s3cli.rb enablelog <bucket> [<targetbucket>]
• s3cli.rb disablelog <bucket>
• s3cli.rb usage|help

• s3cli.rb ls [<bucket>]
• Prints a list of buckets when called with no arguments or prints the objects in the bucket passed as an argument.

• s3cli.rb ls_l [<bucket>]
• Works just like the "s3cli.rb ls" command but with more details. For buckets it prints permissions and the number of objects in the bucket. For objects it prints permission and the size of the object. Permissions are printed in the format "rwrw" where the first set of "rw" indicates read/write permissions for the user and the second set indicates permissions for the public.

• s3cli.rb mkdir <bucket> [public|publicrw|private]
• Creates a new bucket. By default the bucket will be private unless the optional permission argument is specified.

• s3cli.rb put <file1> [... <filen>] <bucket> [public|publicrw|private]
• Copies files to the specified bucket in S3. The created objects will be private by default unless the permission argument was given. Please note that objects do not inherit the permission of the buckets by default. This seems to be a bug either in the aws/s3 gem or in S3 itself. You can give a command like "s3cli.rb put code/* code public" and all the regular files in the code directory will be copied to a bucket named code, if it exists, with public read permissions. The objects will have the same name as that passed to the script.

• s3cli.rb get <bucket> [<file>]
• Copies the specified object or all the objects in the bucket to the location specified by the object name. If the object's name has a directory path embedded in it, "get" will create the directory if it does not exist before writing the file. "get" will overwrite any existing local files with the same name as the S3 object.

• s3cli.rb rm <bucket> <file>
• Removes a specified object from the bucket.

• s3cli.rb rmdir <bucket>
• Deletes a bucket from S3 if it is empty.

• s3cli.rb rmdir_f <bucket>
• Delete a bucket and all the objects in it.

• s3cli.rb geturl [https] <bucket> [<file>] <num<m|h|d|w|y>>
• Gets private url's for the specified object or all the objects in the bucket. The "https" argument if specified will generate secure urls. The last argument specifies the duration the url/urls will be valid for. 'm' stands for minutes, 'h' for hours, 'd' for days, 'w' for weeks and 'y' for years. An example would be "5d" for 5 days.

• s3cli.rb usage|help
• Prints usage information.

• s3cli.rb printlog <bucket>
• Prints the logs associated with a bucket. Log entries will be printed in the format "RemoteIP Date ObjectName". Log entries that were created due to S3 operations initiated by the owner of the bucket will be ignored. For example, if you used this script to access a file in the bucket and you are the owner of that bucket, the entry will not be printed.

• s3cli.rb enablelog <bucket> [<targetbucket>]
• Enables logging for the bucket and redirects them to "targetbucket" if specified. It will return an error if the "targetbucket" does not exist or logging is already enabled.

• s3cli.rb disablelog <bucket>
• Disables logging for the bucket. Returns an error if logging is already disabled.
• 
  TODO

• Better error handling.
• get should have flags similar to -i (interactive) and -f (force)
• Fix commands to have switches. So instead of two commands "ls" and "ls_l", there will be a unix like "ls -l" command.
• Improve ruby usage.