I can’t help but to make a comment on a news I read today at Fox News: Computer Tech Accidentally Erases Info on Alaska’s $38 Billion Oil Fund
“A computer technician at the Alaska Department of Revenue deleted applicant information for an oil-funded sales account — one of state residents’ biggest perks.
While reformatting the disk drive during a routine maintenance check, the technician mistakenly reformatted the backup drive as well and, suddenly, all the data disappeared.
A third line of defense — backup tapes that are updated nightly — were unreadable.
Nobody panicked, but we instantly went into planning for the worst-case scenario,” said Permanent Fund Dividend Division Director Amy Skow, about the computer foul-up in July that ended up costing the department more than $200,000.
Nine months worth of information concerning the yearly payout from the Alaska Permanent Fund was gone:….”
I can totally relate to the experience. I was barely into my first month on the job when a similar thing happened in one of the companies I worked for. Well, the mistake was not worth any near $38B but it cost the company 3 days of down time of the email system. From the experience, I have learned quickly about performance tuning and back-up and recovery procedures.
- The restriction on the level of access- system administrators should only be the ones having the reformat power. You don’t reformat a disk during a routine maintenance check and mistakenly delete the back-up as well.
- The back-up tape was unreadable- for that night yes but what about yesterdays back-up, and the other night’s back up? How come they can only retrieve going back nine months?
- For a system handling $38B worth of account, wouldn’t you think that a robust back-up system is worth the investment? Got to give more respect to the database.
- How come they only plan for the worst case scenario while it’s happening? The plan should have been inplace before it happened. That’s called the back-up and recovery plan. If they did things right, they will only be recovering from yesterday night’s back-up.
- I hope their SOPs are better now.
Well, it’s easy to be a Monday morning quarterback but with proper planning, documentation and implementation all this life changing drama will be avoided. I don’t blame the techician- it’s the system(IT Management) that needs to be corrected.
