How AWS Snowball works?

Snowball service is an offering provided by Amazon that makes use of physical storage devices in transferring large amounts of data at very high speeds between Amazon S3 and user’s data storage location. Snowball helps save time as well as money since it provides a powerful interface which can be used to create jobs, track the status of these jobs and make sure they are completed on time. 

Snowball devices are physically rugged and are protected by the AWS Key Management Service (KMS). They have the ability to provide security so as to protect data when it is being transferred. There are many regional shipping carriers that help in transporting Snowballs between Amazon S3 and user’s onsite data storage location.  

If the amount of data transferred between S3 and onsite data centres is less than 10 TB, Snowball wouldn’t be the right choice, since Snowball is meant to be used to transfer huge amounts of data.  

Features of AWS Snowball 

• In the US regions, AWS provides 80 TB and 50 TB models for data transfer.
• In all other AWS Regions, it provides 50 TB models for data transfer.
• It provides security during data transit as well as when the data is not in transit by enforcing encryption to the data.
• No hardware devices need to be purchased or maintained by the user, since it is facilitated by AWS.
• Jobs can be managed with the help of AWS Snowball Management Console. It can also be programmatically managed with the help of the job management API.
• Local data transfers can be performed between the on-premise data centre and a Snowball. Such transfers can also be performed via a Snowball client, which is a standalone downloadable client.
• The data can be transferred programmatically with the help of Amazon S3 REST API calls and by downloading the Amazon S3 Adapter for Snowball.
• Snowball comes with its own shipping container, and when the Snowball is all set to be shipped, an E Ink display present in it changes to indicate the shipping label of the user.

Note: It is important to understand that Snowball doesn’t provide support to perform internal shipping or shipping between AWS regions which are present outside the US regions.  

Prerequisites to use AWS Snowball

Before data is transferred into Amazon S3 with the help of Snowball, the following facilities have to be active.  

• An AWS account has to be created and an admin user in AWS IAM (Identity and Access Management) has to be created.  
• When data needs to be imported, the below steps have to be adhered to: 
• The files and folders which have to be transferred have to be named in accordance to the Amazon S3 Object Key Naming Guidelines. If a folder or file doesn’t meet the guidelines provided, it will not be imported into Amazon S3.  
• It is important to plan beforehand the type of data which needs to be imported into Amazon S3.  
• When data needs to be exported, the below steps have to be adhered to: 
  • Know and understand what kind of data would be exported when the ‘job’ is being created.  
  • Files which have a colon (:) in the filename have to be changed in Amazon S3 based on the guidelines for naming the files and folders. This has to be done before a job is created to export the files. If the files contain a colon, the export job fails to export data to Microsoft Windows Server.  

Snowball makes use of AWS Snowball Management Console and job management API to create and manage jobs. To perform transfers locally, Snowball user’s client or AWS S3 adapter for Snowball.  

Data import working

Below is an image that shows how data import works in Snowball service:

Image credit: aws.amazon.com 

• Every import job in the service makes use of a single Snowball appliance/device. Once a job has been created in the AWS Snowball Management Console or the job management API, Amazon ships a Snowball to the user.  
• This Snowball arrives in a few days, and the user can connect the Snowball to their network and use it to transfer data which needs to be imported into Amazon S3 from their storage location to the Snowball using the Snowball client or the Amazon S3 Adapter for Snowball.  
• Once the data transfer is complete, the Snowball is shipped back to the AWS, and the data present in Snowball is imported to Amazon S3.  

Data export working

Below is an image that shows how data export works in Snowball service:

Image credit: aws.amazon.com

• An export job can use any number of Snowball appliances/devices. Once the export job has been created in the AWS Snowball Management Console or using the job management API, a listing operation is initiated in Amazon S3.  
• This listing operation splits the export job into parts and every job part can take up to about 80 TB in size or 50 TB in size depending on the AWS region.  
• Every job part will have exactly one Snowball associated with it.  
• Once all the job parts have been created, the first job enters the phase of ‘preparing Snowball’.
• After this, data starts to get exported to the Snowball. Exporting data typically takes about one business day, but can take longer at times.  
• Once all the data has been exported, AWS makes sure that the Snowball is ready to be picked up by the user’s region’s carrier.
• When this Snowball is delivered to the user’s data centre in a few days, it can be connected to the network and data can be transferred to the user’s servers using the Snowball client or the Amazon S3 Adapter for Snowball.  
• Once this data transfer is complete, the Snowball has to be shipped back to AWS. Once the Snowball reaches AWS, they perform a complete erasure which deletes all the data. This indicates that the export job is completed successfully. This erasure follows the National Institute of Standards and Technology 800-88 standards.  

Conclusion 

In this post, we understood how AWS Snowball works in transferring data at very high speeds between Amazon S3 and user’s on-premise data centre.