Simple DynamoDB Table Backup & Restore

When migrating AWS environments there is a typical need to migrate data. In terms of data migration & backup there are many very nice options for DynamoDB today, including

• Amazon S3 and AWS Glue
• Data Pipeline
• Amazon EMR

They are serverless and mostly don’t require any coding. They support Dynamo Streams and should be generally used during warm/hot no-downtime migrations.

But what should we do if the downtime requirements are much more liberal and the data amount is rather small? Do we really need to build a complex system that involves multiple AWS services for a simple migration that can be done with 1 minute cutover?

I figured that the answer should be “Yes”. I was looking at options and found bchew/dynamodump, which is a small python utility written with boto3. The usage is rather simple.

Installation

pip install dynamodump

Simple Example

dynamodump -p infra-legacy -m backup -r us-west-1 -s testTable

dynamodump -p infra-prod -m restore -r us-west-1 -s testTable

Notes & Recommendations

• Use -p parameter to specify profile, so there is no mixup on source and destination
• Keep --skipThroughputUpdate parameter in mind, if sometimes there is an issue with automatic throughput settings.
• Keep network bandwith capabilities in mind, and if your data amounts are substantial, use an ec2 instance to reduce your TTR.