Today I almost FUBAR’ed a customer their production instance because of forgetting one small thing.
I did a data refresh from production to staging, but in my haste forgot to update the database connection string to the staging database.
When I launched the application, it became quickly clear that something was wrong. But luckily not in the production environment, as past me had the foresight that sometimes you just do stupid things.
The staging application started spewing a lot of errors because it could not connect to the database.
I try to make it a habit to invalidate production hostnames in the hosts’ file on staging machines like so:
## There is absolutely no reason to be able to reach these hosts from staging# If you need to copy files, do it from production to staging, not the other way around# This should be an unresolvable address (which localhost isn't!) for the application# links update to work.#192.168.7.8 jira.customer.com productiondb.customer.com |
The comment says it all, really, you redirect the hostname to an IP that doesn’t exist.
Even if that protection had not been in place or invalidated because somebody didn’t update the hosts’ file when a hostname changed, I would have had a second layer of protection in place: the user that the application uses to connect to the MySQL database is only allowed to connect from the production machine (do not use % as the allowed hostname!).
And even if somebody had messed with the allowed hostname, there is a third layer in place: the user that is connecting has different passwords on the staging and production database.