Power Outage Knocks Wikipedia Offline

The free online encyclopedia Wikipedia has been knocked offline by a power outage in its data center. While the servers hosting the site were down only a short time, much of the site's content remained offline as Wikimedia staff worked on properly restoring data from MySQL databases.

"At about 14:15 PST some circuit breakers were tripped in the colocation facility where our servers are housed," WikiPedia reported in a message to users. "Although the facility has a well-stocked generator, this took out power to places inside the facility, including the switch that connects us to the network and all our servers. The sticky point is the database servers, where all the important stuff is."

"Although we use MySQL's transactional InnoDB tables, they can still sometimes be left in an unrecoverable state," WikiPedia staff added. "Attempting to bring up the master database and one of the slaves immediately after the downtime showed corruption in parts of the database."

Last month the LiveJournal weblog service suffered an extended outage in similar circumstances, with a data center power triggering a lengthy recovery of large volumes of data in MySQL databases.

For both LiveJournal and Wikipedia, the recovery process coincided with posts on Slashdot, which have been known to overwhelm sites with traffic. Not to fear - the Wikipedia entry on the Slashdot effect was online and available last night, even as other parts of Wikipedia remained offline.