It’s every system administrator’s nightmare. The Unix command rm is powerful and wielded properly can be a very useful tool. But used improperly, it can devour you. That is what has happened at a web hosting provider. There, the system administrator made a small little mistake on the command line, but the impact was completely devastating.
The system administrator had posted in ServerFault about recovering from a rm -rf / mistake. For those unfamiliar, this command essentially tells the server to delete every file, in every directory and their subdirectories, from every accessible storage space. In short, it wipes out the entire server. It wipes out any storage connected to it.
It’s total annihilation.
If your company runs entirely in this one server, it’s entirely gone.
That seems to be pretty much what happened at this company. They did have many servers, but automated tools repeated the erroneous command on every server. They had off-site remote backups. But the backups were connected to the servers at that point in time, and so the erroneous command also wiped them out.
While the system administrator perhaps still harbours a glimmer of hope that someone on ServerFault might somehow have some magic solution, the respondents aren’t totally helpful. In fact, at this time, ServerFault has locked the thread because of the high amount of off-topic comments generated.
Like, this one, for example.
You’re going out of business. You don’t need technical advice, you need to call your lawyer. — Michael Hampton
With that one command, his company has basically been deleted.
This is a sobering reminder to all system administrators. With great power comes great responsibility. We all make mistakes. Unfortunately, some errors lead to complete devastation. Hence it is so important to be extra cautious, to have all manner of safeguards, procedures, and workflows, to prevent or mitigate the impact of such errors. Backups and contingency plans too, of course, but we hope never to have to resort to them.
View Comment Policy