Simple DynamoDB Table Backup & Restore
When migrating AWS environments there is a typical need to migrate data. In terms of data migration & backup there are many very nice options for DynamoDB today, including
- Amazon S3 and AWS Glue
- Data Pipeline
- Amazon EMR
They are serverless and mostly don’t require any coding. They support Dynamo Streams and should be generally used during warm/hot no-downtime migrations.
But what should we do if the downtime requirements are much more liberal and the data amount is rather small? Do we really need to build a complex system that involves multiple AWS services for a simple migration that can be done with 1 minute cutover?
I figured that the answer should be “Yes”. I was looking at options and found bchew/dynamodump, which is a small python utility written with boto3. The usage is rather simple.
pip install dynamodump
dynamodump -p infra-legacy -m backup -r us-west-1 -s testTable dynamodump -p infra-prod -m restore -r us-west-1 -s testTable
-pparameter to specify profile, so there is no mixup on source and destination
--skipThroughputUpdateparameter in mind, if sometimes there is an issue with automatic throughput settings.
- Keep network bandwith capabilities in mind, and if your data amounts are substantial, use an ec2 instance to reduce your TTR.