The collaborative coding site scrambled to withstand the opening salvo from what researchers dubbed China’s Great Cannon. But CEO Chris Wanstrath says that was just the beginning. To survive, startups must surmount challenges like product development, funding negotiations and cash flow. GitHub CEO Chris Wanstrath can add a very different challenge to his list: a sustained five-day network attack that some say marked the beginning of a new, more aggressive chapter in China’s relations with the outside computing world. GitHub’s business, founded in 2008, is all about letting programmers work together. It offers a place where individual coders can contribute to each other’s software projects, and where companies like Google, Facebook and Twitter can share work through the collaborative open-source movement. But on March 26, two organizations with GitHub accounts came under attack. Attacks on GitHub are common, though it can be nearly impossible to figure out their origins, Wanstrath said during an interview here at the company’s Merge conference. Even teenagers flexing their online muscles can launch an attack by buying access to a collection of machines. But this recent GitHub attack one was the worst in the company’s history. The company’s seven-person response team worked around the clock in a cat-and-mouse game to keep GitHub running even as the attackers shifted from one type of attack to another. Those two targeted GitHub sites were GreatFire.org, a nonprofit organization that tries to help people bypass Chinese censorship, and the Chinese New York Times, according to an analysis of the attack by network security software firm Netresec. But it hurt all of GitHub’s operations. That’s because it was a distributed denial-of-service (DDOS) attack, where countless computers around the world overwhelmed GitHub’s servers to the point where they couldn’t provide the online service they’re supposed to provide. Researchers dubbed the attack the Great Cannon. The Great Firewall of China has been around for years, letting the government block access to sites it doesn’t want its Chinese residents seeing, but the Great Cannon serves an offensive rather than defense purpose, the researchers at the University of Toronto, University of California and Princeton University wrote. When people visited innocent Web pages, the attacker’s servers would replace website code with malicious code that would direct their browsers to ceaselessly reload the GitHub pages. “The Cannon manipulates the traffic of bystander’ systems outside China, silently programming their browsers to create a massive DDOS attack,” the researchers said. The Chinese system could work similarly to one run by the US National Security Agency and its British counterpart, Government Communications Headquarters, according to documents leaked by former NSA contractor Edward Snowden. These programs, called Quantum and Foxacid, appeared to target the anonymous communication technology called Tor and employees at Belgian telecommunications company Belgacom, according to security expert Bruce Schneier and Der Spiegel, a German news publication. Wanstrath sat down with CNET’s Stephen Shankland to discuss the GitHub attack. The following is an edited transcript of their conversation. What was your first inkling that you were under attack? Wanstrath: A traffic spike. We started to get an unusual amount of traffic. It was coming from all over the world — were we on Oprah? Then we realized people’s phones or computers were getting hijacked to load GitHub. We saw the man-on-the-side attack. But that was just was the first attack of a series. Wanstrath: Yes. It was a mix of new stuff and boring stuff. The nature of the first attack was novel. After that we saw other attacks that were traditional, like SYN floods. In five days, we saw 18 or 20 attacks. How often are you attacked ordinarily? Wanstrath: Once a month, if not more. We’ve got monitoring. We have a good incident response program set up. When there’s an attack profile, you get paged. The main event of a DDOS is overwhelming the network with traffic. When you get a million requests and they’re exactly the same in one second, that’s a DDOS. We have automated systems, then an ops team on the network around the clock. So was somebody trying to send a message? Wanstrath: Of course. I just don’t know who the message was for. I’m not even sure the message is to us. You don’t need to be a state government to run this sort of attack. Sometimes it’s teenagers fighting over message boards. If it was from China, is there an easier way to target GreatFire and the New York Times than launching a five-day attack? Wanstrath: Sure. That’s why it’s confusing to conclude it came from China. In China, the New York Times is blocked, the Wall Street Journal is blocked. China blocks [lots] of websites. And after five days they chose to disengage? Did you vanquish the enemy? Wanstrath: It was an ongoing battle. We successfully mitigated some of their attacks. Even though we were winning, we were fighting the whole time. There was a lot of press about it, which may have contributed to the disengagement. What’s frustrating is there was no ransom note — no request for anything. Just an attack. What did it do to your business? Wanstrath: The outages are frustrating. We never went totally down, but people had errors. It interrupted people’s workflows. At GitHub, people were up all weekend. So is this a badge of honor? A sign that you’ve arrived? Wanstrath: It’s hard to feel that way when there are real people trying to do real work with GitHub. If this is what arriving is like, this isn’t what we signed up for. We’ve been attacked for awhile. We have defenses. But GitHub two or three years ago would not have successfully mitigated this attack. You can imagine a smaller company just falling over. What did you learn? Have you changed any technology or policies? Wanstrath: We learned a lot on a technical level. The DDOS is such a cat-and-mouse game. We can’t share broadly with the technology community to say here’s how to protect yourselves, though. It’s like bacteria. If the attackers know what we do, then they’ll stop doing that attack. Now, they don’t know what we know. Did you talk to the US government about the attack? Wanstrath: We can’t say it really has a China component because we can’t prove anything. We can’t really ask for help for anyone. I’m not sure what would have happened if this had lasted a month. Source: http://www.cnet.com/au/news/how-startup-github-survived-a-massive-five-day-network-attack-q-a/
Original post:
How startup GitHub survived a massive five-day DDoS attack