Outage at Revera's Auckland datacentre
LATEST NEWS
SUBSCRIBE
Computerworld is New Zealand's only specialised information systems fortnightly. Subscribe now for $100 (23 issues) and save more than 37% off the cover price!
SIGN UP
Test battery fault causes 90 minute outage at Auckland datacentre
By Randal Jackson | Wellington | Wednesday, 5 September, 2012 | 31 Comments
Part of Revera’s datacentre at Auckland’s William Pickering Drive lost power yesterday evening during scheduled preventative UPS maintenance.
The outage occurred around 5.30pm.
A test battery fault triggered the release of suppression gas, causing a UPS shutdown, resulting in the loss of power to one datacentre suite in the facility.
Revera general manager Robin Cockayne says it was not a failure of any critical power system or backup power system, rather a battery fault that has since been remedied.
He says Revera immediately initiated a start-up sequence and restored power in less than 60 seconds. Technicians then undertook remedial activity to return affected production environments to normal service.
The majority of the affected systems were fully operational by 7pm.
Comments
Stop complaining
If we woke up and learned one morning from reading the newspaper that 12 jumbo jets filled with children had crashed, leaving no survivors, the world would be outraged and distraught. Yet, each day, a child somewhere in the world is dying every 8 seconds from drinking contaminated water - the equivalent of 12 jumbo jets of children dying per day. Now Revera was down less than 1 hour, wow, that is really is worthy of debate. Get a life.
Posted by Take the red pill at 15:40:13 on September 6, 2012
Posted by Take the red pill at 15:40:13 on September 6, 2012
Outage
I've worked with Roger for a number of years and I have no doubt the outage was unexpected and unfortunate, and the problem could occur in any Data Centre.
The reality is outages in Data Centres occur - and the real problem is when client don't plan for - or expect - them to occur.
That's when you see significant outages caused by equipment, suppliers response (or lack of) and IT staff struggling to recover.
Plan for it, and you won't be caught out or surprised, or find yourself trying to explain why there was an outage in the first place.
Posted by Jeremy Strachan at 22:32:35 on September 6, 2012
The reality is outages in Data Centres occur - and the real problem is when client don't plan for - or expect - them to occur.
That's when you see significant outages caused by equipment, suppliers response (or lack of) and IT staff struggling to recover.
Plan for it, and you won't be caught out or surprised, or find yourself trying to explain why there was an outage in the first place.
Posted by Jeremy Strachan at 22:32:35 on September 6, 2012
Stop complaining
If a tree falls in a forest and there is no-one around to hear it, does it make the same sound as one hand clapping?
Posted by Anonymous at 17:09:46 on September 6, 2012
Posted by Anonymous at 17:09:46 on September 6, 2012
Tier 4?
There absolutely are Tier 4 data centres worldwide
Posted by John Holley at 8:32:45 on September 7, 2012
Posted by John Holley at 8:32:45 on September 7, 2012
questionable
"A test battery fault triggered the release of suppression gas, causing a UPS shutdown"
1 most computer rooms /data centres with suppression gas require both heat and smoke detectors to operate before the release of gas !! so did they have a battery fire ?
2 why would you shut down a UPS on a release of gas ?
3 most racks would/should be feed from 2 independant UPS's and equipment in the racks with dual / redundant PSU so why did any equpment go down
Posted by Anonymous at 12:58:38 on September 6, 2012
1 most computer rooms /data centres with suppression gas require both heat and smoke detectors to operate before the release of gas !! so did they have a battery fire ?
2 why would you shut down a UPS on a release of gas ?
3 most racks would/should be feed from 2 independant UPS's and equipment in the racks with dual / redundant PSU so why did any equpment go down
Posted by Anonymous at 12:58:38 on September 6, 2012
questionable
1. Not true, most gas systems will go off on smoke but require two detectors to be hit
2. and 3. yeah, really odd eh ? - maybe someone panic ?
Unless the building has a power shut down on Fire Activation ? Wow, >>shoot the designer if that is the case ??
Posted by Anonymous at 17:23:30 on September 6, 2012
2. and 3. yeah, really odd eh ? - maybe someone panic ?
Unless the building has a power shut down on Fire Activation ? Wow, >>shoot the designer if that is the case ??
Posted by Anonymous at 17:23:30 on September 6, 2012
something refreshing and locally unique
One thing that can be said here is at least revera fronted up with facts. Drury's little tanty wasn't about uptime,it was about twitter presence. Not even the public cloud champion can claim better uptime than the local providers. I'm aware of no less that half a dozen outages across the big 4 this year alone. They may have tweeted up dates but I bet none picked up the phone to call their clients. A shame to see a local provider lose a decade long record, but its the nature of the business and I'll always go with a one in ten record than 4 in 1!
Posted by MF at 10:35:34 on September 6, 2012
Posted by MF at 10:35:34 on September 6, 2012
putting it in perspective
Its a pity there are so many negative comments here as Revera have an excellent track record. But outages at data centres happen. In the last few years Google and Microsoft have both had multi day cloud outages while locally IBM's newton facility had an outage which caused huge for Air NZ for 8 hrs and prompted their new high brook data centre.
Posted by henareho at 10:10:28 on September 6, 2012
Posted by henareho at 10:10:28 on September 6, 2012
What is tier 3 etc.?
A lot of comments about uptime etc. If you are in a tier 3 data centre you need to plan on around 90 minutes of outages a year based on industry standards.
See http://en.wikipedia.org/wiki/Data_center for a simple explanation or download the paper at http://www.uptimeinstitute.com/component/docman/doc_download/5-tiers-standard-topology for more detailed information.
From what I see there is no need for an alternative national power grid e.g. separate power feeds to a data centre, for it to be tier 4. How a data centre deals with external power failures, which are to be expected and planned for, is the question.
Posted by John Holley at 10:09:56 on September 6, 2012
See http://en.wikipedia.org/wiki/Data_center for a simple explanation or download the paper at http://www.uptimeinstitute.com/component/docman/doc_download/5-tiers-standard-topology for more detailed information.
From what I see there is no need for an alternative national power grid e.g. separate power feeds to a data centre, for it to be tier 4. How a data centre deals with external power failures, which are to be expected and planned for, is the question.
Posted by John Holley at 10:09:56 on September 6, 2012
MOST POPULAR
Social Media @Computerworld NZ
Computerworld NZ has now reached LinkedIn! Join to expand your networks and meet others interested in information systems.
