Another Gmail Outage

Posted: December 12, 2012

Gmail embarrassed by another email outage. The outage was caused by a software update.

Less than a week after Google announced they will begin charging small businesses for Google Apps, a Gmail outage occurs.  Even a company as large as Google is susceptible to outages. 

According to

What would life be like if Google crashed? People got an unexpected opportunity to experience Google withdrawal on Monday morning, when a configuration change to piece of infrastructure known as a “load balancer” knocked Gmail offline for total of 18 harrowing minutes.

The glitch affected both consumer and corporate customers and came at a particularly inopportune time: when millions of West Coast workers were logging in to their email and getting started with their day.

According to an explanation Google published Tuesday afternoon, there were two outages. The first lasted six minutes, from 8:54 a.m. to 9:00 a.m. Pacific Time. The second lasted from 9:04 a.m. to 9:16 a.m. Pacific Time.

At 10:10 a.m. Pacific Time, Google announced that the issue was resolved and apologized for any inconvenience. “Please rest assured that system reliability is a top priority at Google, and we are making continuous improvements to make our systems better,” the company said in a statement.

In addition to Gmail, the outage also affected Google Drive, Google Chat, Google Calendar, Google Play and Google Chrome Sync.

Responses to the outage ranged from panic to snark as engineers made cracks about the wisdom of updating your infrastructure first thing on Monday morning. “Google Shuts Down Gmail for Two Hours To Show Its Immense Power,” observed The Onion.

“We can tell Gmail is down because we just got a smoke signal from a Nigerian prince,” Comedy Central quipped on Twitter.

According to a thread in one of Google’s developer forums, the incident stemmed from “a faulty load balancing configuration change” that was made to a piece of infrastructure known as the “Chrome Sync Server.” Unhappily for Google, “the change was to a core piece of infrastructure that many services at Google depend on.”

Load balancers are responsible for routing the requests made by Google’s millions of users to data centers around the world. In this case, “a bug in the software update caused it to incorrectly interpret a portion of Google data centers as being unavailable,” Google said in a an incident report to customers of Google Apps.

In other words, it’s the kind of error that could happen to any cloud service, and it should serve as a reminder that even Google, for all its engineering prowess, is fallible.

What should you do if you’re a Google user? Make a backup of your Google data, just as you would backup any storage device you used at home. Here’s some information about how to use Google Takeout, one of the world’s easiest backup services.