How tech giants spread open source programming love
That may be the case, yet the open source language is increasingly being adopted by enterprises around the world for building applications at large scale.
The story is similar with Erlang. Originally a proprietary language developed by Swedish telecommunications company Ericsson for handling massive volumes of telephone switching data on its hardware, Erlang was open sourced and is now rapidly gaining popularity for large-scale applications.
And there's more. Facebook developed GraphQL and BigPipe technologies in response to the particular challenges it faces running a social network processing hundreds of billions of API calls a day for over 1.5 billion active members. Today these technologies have been open sourced, and are used by the likes of content management project Drupal to make its mobile web pages load faster.
You may have noticed a common theme here: Programming languages and technologies that were developed by industry and Internet giants – specifically to meet the unique challenges they faced operating at massive scale – have been open sourced and are now being adopted by regular-sized enterprises for everyday use.
Part of the reason for this is a natural technology trickle-down effect, according to Mark Driver, a research director at Gartner. "Today's leading edge super high tech is tomorrow’s standard product," he says. "Also, large companies (like Google and Facebook) understand the collaborative nature of open computing and the dynamics that drive the Internet. So it's natural that they share these technologies and strengthen the industry around them."
It may also be that companies like Google share their technology for more self-centered reasons. For example, in November Google open sourced its TensorFlow artificial intelligence and machine learning engine. This needs to be fed huge amounts of data to work effectively.
[Related: 8 key open source software foundations (and what makes them key)]
By giving away its TensorFlow technology, Google’s allowing everyone to benefit from the technology and any improvements that are made to the code. But as the owner of vast amounts of data gathered through the many services it offers, Google is in a position to benefit more than most from improvements made to the technology by the open source community.
There's also a considerable marketing benefit in promoting a language like Go that's available to all, Driver believes. "Sponsoring a project like Go can help Google drive the recruitment of engineers," he says. "It can lead to a virtuous circle of innovation."
The trickle-down effect is particularly important because the challenges that Google, Facebook and other Internet giants faced a few years ago building large-scale applications with huge numbers of concurrent users are exactly the ones faced today by thousands of established companies and startups offering web- or mobile-based applications to large numbers of their customers.
These companies are discovering that many of the problems they’re facing have already been solved.
Acquia is just such a company. It uses Go for its software-as-a-service offering that provides enterprise services for the Drupal content management system. "More and more companies are becoming data companies, handling data from customers, mobile devices and so on,” says Christian Yates, a vice president at Acquia.
"These are exactly the things that the largest Internet companies had to deal with in the past and they built software to cope with it. They had the same issues of high degrees of concurrency and concern for latency and performance that we have today."
He says that his company did a bake-off involving different programming languages and found Go to be the fastest. The fact that it is open source also allowed coding teams to get up to speed and start innovating quickly, he adds.
"Broad adoption in the marketplace, and visibility into source code and how it is used certainly helps developers get productive much more quickly," says Yates. "And we have leveraged tools from Etsy and other Internet companies to help up manage our servers."
When Ericsson developed Erlang back in the mid-1980s, the World Wide Web had not yet been invented. But it turns out that it’s ideal for many Web applications, according to Gartner's Mark Driver.
"Erlang was for massive volumes of switching data, and it was well ahead of its time. Now many companies are building message-oriented, massive-scale high volume applications, and Erlang is perfect for that," he says. As an example, the WhatsApp messaging platform that Facebook acquired in February 2014 uses Erlang to support over 900 million users of the service.
[Related: How to make money from open source software]
Another such company is Bet365, a U.K.-based online gambling company. Today it has 7 million or more concurrent users of its services at peak times, with traffic growing by about 20 percent every year. The company originally built its online platform using Java and .Net, but struggled to keep scaling up. It also faced time-to-market challenges for new services, according to Chandrashekhar Mullaparthi, the company's principal software architect. This led to a switch to Erlang on which to build its platform.
"Erlang is ideal for us," Mullaparthi says. "It may have been developed by a telecoms company, but the challenges faced by a consumer-facing site like ours are actually very similar. For building an app that needs to scale, has high concurrency and needs "Five nines" availability [i.e., 99.999 percent], I can't see anything else better we could use."
With an increasing number of companies offering large-scale Internet-based applications, the main barrier to adoption of languages that originate from behemoths like Facebook, Google or Ericsson is a shortage of developers with appropriate skills.
That's been the experience of Bet365. Although Erlang is a mature language, it's only recently that its suitability for many online platforms has been broadly recognized. That means that recruiting Erlang developers has not been easy, Mullaparthi says, and allowances have had to be made.
"It would certainly have been hard if I had said that I wanted everyone in the office every day," he concludes. "But we have not had a problem once we realized that. Now we work with developers who work remotely elsewhere in the country. Also we are taking on fresh graduates and training them ourselves," he says.