Pubmatic cloud chief on his six-figure AWS hidden-cost horror show
Udy Gold, speaking at Openstack Summit 2016, likened his experience with public cloud to the "most expensive taxi ride" of his life - when in India he knew he was being ripped off for a rickshaw ride but handed over the $20 anyway. It led to a situation so desperate he was hiding from his CFO.
When Pubmatic first signed up to AWS the thinking was perfectly logical: the business had a situation where development needed new machines and new hardware. With a CFO withholding the level of budget this would require, Gold wondered: how do we do it without buying new hardware And so the firm signed up to the public cloud.
He reminded the room that the public cloud does not equal the data centre: it's CPU for hire. "So if you need a CPU for 24 hours a day, then the public cloud is going to bite you with the cost," he said.
Then there's the costs the business would have to take on anyway: people who understand what the company's doing, resource management, system administrators, network monitoring. And there are other challenges, like troubleshooting latency issues, machines dropping, contacting the provider, for example.
"The cost is unpredictable," Gold said. "It can be very expensive and very difficult to control."
"We basically put money down the drain."
Although Gold didn't reveal the exact figures, he did tell the room the ratio. The project on AWS ended up costing 18 times the original budget. But by the time the invoice arrived it was too late. The company was heavily invested with over 1,000 production nodes in the public cloud - so of course the cost skyrocketed.
"When I came to the office I had to go through the back door and climb under the table so the CFO wouldn't see me," Gold said. "I'm not kidding. It was unpleasant."
This unsustainable cost reached six figures and sometimes close to seven - a month. It fluctuated, and there just was not a lot Gold could do. It turned out the vendor didn't want to negotiate.
The CFO was asking: "What do we need to do to get out of it" And he was back to square one. Pubmatic needed hardware, network, management and control.
So the company set about stringently defining its requirements, decided to use its own existing hardware, centralise management, and it wanted strong vendor support. The integration, it was decided, would be as simple as possible.
Enter Platform 9, one of the earlier Openstack vendors. Gold said: "I explained my requirements, they were attentive. They allowed me to work on their roadmap, they were hardware agnostic - I didn't have to take down a single machine to bring it into their environment."
Pubmatic shifted to this Openstack-based private cloud in its own data centres, and is now running a hybrid data centre where it largely runs on private cloud but relies on public for capacity spikes. And everything is managed from the same place.
"So, lesson learned," Gold lamented. "I experienced a lot of sleepless nights. And it's not black or white. Others will tell you public is the best thing that's ever happened."
"But remember to be realistic with your expectations and your estimates. I know that every project that's being given to me is going to start in a certain way, but will not end in the same way. There's always requirement change, add-ons, whatever it is.
"Always prepare yourself with enough of a buffer not to be in a situation where you said the project is going to be $50,000 a month and you end up with a $200,000 invoice, because you are the one who's responsible for that.
"Vendors like to talk about success stories in public cloud and there are very good ones. But they might not work for you."