Blog
The hard truth of RDS migrations is that they never go off without a hitch, but that doesn’t mean they have to be a train-wreck. Mission has migrated countless companies to RDS, and we’ve learned a lot about what can go wrong and how to avoid those pitfalls—or at least prepare for them to minimize the hassle and interruption to your project schedule.
And that’s the most important lesson there is: Plan for things NOT to go 100% according to plan. The key to a smooth migration is not to expect everything to go perfectly; it’s being able to roll with the punches, get ahead of things where you can and react efficiently when something does go wrong. Problems don’t mean anyone screwed up; they’re an unavoidable part of migrating a database to the cloud.
With that in mind, we’ve got some tips to share with you that should help you get ahead of the most likely problem areas.
RDS Tip #1: Treat Your Data Like It’s Your Most Valuable Asset (Because It Probably Is!)
For many businesses, their data is their most valuable asset, yet many companies cut corners with their backups and backup processes. I mean, I get it: it’s an easy thing to put off, because you have more pressing matters to attend to (And you’re planning a migration! You’re busy!). But consider the range of negative outcomes in a migration, and I’d wager you’ll have a hard time coming up with something worse than losing or corrupting your company’s data.
So before you do anything else, make sure your backups are rock solid. For every database we manage outside of RDS, we check backups at least once a week manually to ensure they haven’t failed. Before you embark on a migration, make sure that your backups have been tested and that you’re monitoring backup procedures regularly. Remember: backups are essential to every business, but a flakey, untested backup has little real value.
RDS Tip #2: Give Yourself Time to Migrate
With backups locked in, you can start to think about the actual migration to RDS. The first thing to consider is how long the process, start to finish, is going to take. Based on our experience, you should plan for at least four weeks start to finish—and that’s assuming you know exactly what you’re doing every step of the way and you have an unusually smooth ride. So give yourself a break and plan for longer from the outset so you don’t feel like you’re behind schedule on day 1.
RDS Tip #3: More Data = More Time Migrating (Maybe A LOT More)
One of the most important questions to consider while you’re planning your RDS migration is how big is your database. The less you need to migrate, the easier (and quicker) the entire process ends up being. That may seem obvious, but it’s really important to remember, because you may be able to help yourself out a little.
Consider this real-world example of why it’s important to think about the size of your database. I have a friend who works at a major online retailer that was migrating to RDS. Their database had ballooned to over a terabyte. You can see why: their entire product catalog was there, including archived items they no longer sold. They were really struggling with the large database, and with a limited amount of time to get migrated to RDS before the holiday shopping season, they were under the gun. (Add that to your list of RDS nightmare scenarios: you work for a retailer and you can’t get the database in place to support holiday shopping. Definitely not a good outcome!)
The solution for my friend was a simple one: migrate only what you need. By removing discontinued items from the database, they were able to reduce the size of the database by 75%! Putting the database on that weight loss plan not only meant they hit the migration deadline with time to spare, but they also improved the overall website performance since searches took far less time!
RDS Tip #4: Minimize Downtime But Set Realistic Goals
If you ask someone NOT on your tech team how much downtime you can afford during the migration, by far the most common answer you’ll hear is “None.” Okay, yeah, I get it: we all want to migrate with zero downtime.
Zero-downtime migrations are definitely doable, but they’re also definitely more complicated. If you can afford some downtime, a good approach is to plan for it realistically. As a general rule of thumb, the bigger the dataset, the more downtime you should expect and plan for. Export/Import is the simplest way to migrate the data, but you should test-run the export/import and time it to see how long the process takes. That will give you a reasonable initial downtime estimate. Again, don’t make your plans off of best-case scenarios; be reasonable in the expectations you set.
If that amount of downtime isn’t an option, then you need to configure a live streaming replication cutover. There are plenty of tools and options for this process—MySQL, Postgres, bucardo, Oracle Data Pump, to name a few—and if the native option doesn’t work properly, you can always fall back to Amazon’s Database Migration Service (DMS, for short).
One last note on the import: Make sure you turn off Multi-AZ and all your backups and replication—the last thing you want to do is deploy a dysfunctional database across your entire cloud infrastructure! Get it imported and then turn all that stuff back on.
RDS TIp #5: Version Considerations and Incompatibilities
Migrating into Aurora has some quirks, and a major one is version consideration. We always recommend staying on major versions when doing a migration when possible. If, for whatever reason, you can’t do that, you should expect to spend additional time and effort on the migration due to idiosyncrasies related to versions. Or better yet, upgrade (or downgrade) your database to match what is in Aurora prior to starting the migration.
When migrating from MySQL to RDS using native MySQL replication, you’ll need to set your bin log configuration to line up with what RDS requires. More importantly, make sure that your backup point is current in the event that you need to do a rollback. If you’re using Amazon DMS to perform the migration, make sure that any additional source configuration changes are implemented before you start the migration—like bin log format and expire log numbers, which do require a restart. Otherwise, you’ll be racing the clock to implement these changes after a failed migration.
Many older workarounds and temporary fixes do not work with RDS. For example, super privileges are not allowed by RDS. If your mysqldump contains commands that require super user privileges, you’ll need to update that backup file in order to get it into RDS.
RDS Tip #6: AWS DMS and the Quirks of Datatypes
When it works right, AWS’s Database Migration Service (DMS) is a godsend, but it is one of the more finicky tools released by AWS. DMS can be used to move from one database platform to another, and it excels with relatively simple datasets that have a small number of data types. Be warned, though: DMS may end up converting your column datatypes when it attempts to standardize them.
DMS can really save some time—but only when/if it works. AWS is working on removing the need for datatype transformations, but for now, you’ll need to consult the master list before you start migrating— and you need to regularly check for updates to the master list. If your datatype isn’t on the list, you should probably stay away from DMS altogether, unless you are confident those changes will not impact your application functionality.
RDS Tip #7: Look Into Existing Licensing, Tools and Third Party Product Roadblocks
Without proper planning, licensing issues will turn into a major migration roadblock or a possibly-significant unexpected cost. Licensing is an expensive investment, and companies like Oracle frankly aren’t making it easy to migrate to the cloud. We strongly suggest you BYOL (bring your own license) when you’re using Oracle or Microsoft RDS database types if you’ve already invested instead of wasting that investment and purchasing new licenses. Definitely take the time to do a full TCO (total cost of ownership) breakdown to see what licensing approach makes the most financial sense.
One note about running applications or tools in the cloud: Because RDS has no operating system on the database server, you can’t run any applications or tools directly in the database. Instead, you’ll need to configure them elsewhere, like in an EC2 instance. The same is true for any procedures you have that reference those executables. This is most often the case with Oracle and Microsoft databases.
RDS Tip #8: Get the RDS Support You Need
One of the biggest benefits of RDS and DMS is that they are both managed products from Amazon Web Services. A lot of first time migrators attempt to get by with just AWS developer support, but in our experience, it’s very unlikely that that will be sufficient. AWS Business Level Support is well worth the additional investment, and if you just go with it from the outset, you’ll save yourself a lot of time and headache.
So, is RDS worth it?
Absolutely. Yes. 100%.
There’s no question that migrating to RDS has a learning curve, but migrating to RDS will deliver very real benefits across your organization. With RDS, you can expect to see improved total cost of ownership (TCO), improved uptime, increased performance and less expensive backups of your databases. The amount of effort you would be spending on keeping a database up and running, properly backed-up, syncing to replicas and monitoring it all far outweighs the additional cost of RDS over an EC2 instance running the database software you may be more familiar with. Take the time to climb the learning curve, and you won’t regret it!
Get started on your RDS migration by scheduling a 30-minute SA On-Demand where you can talk to me or one of our other engineers about what steps you need to take to get ready to migrate – just let them know Kiril sent you!
Go Back
Share this post