Why are we racing to DevOps
There’s just one question: Just what is DevOps Some people characterize it as refashioned agile development practices; others describe it as new supporting tooling that automates the process of assembling, deploying, and operating applications; still others refer to it as a cultural change. In other words, it’s an ambiguous term that can be interpreted in many different ways.
I have been working in and around DevOps for the past several years, and sometimes despair at the endless discussions on the topic; it often seems people would rather discuss DevOps than actually do something to implement it.
Nevertheless, despite this ambiguity, there’s something there – something important. And understanding what’s driving the race to DevOps and what it takes to participate (or, indeed to win) the race is critical to success in the future – and not just success in the sense of being a top decile IT organization, but success in the sense of a company being able to compete in the ever-evolving global economy.
Given my experience with DevOps and how important it is to the future of how IT operates, I pay a lot of attention when I come across insight on the topic. Recently, I encountered an interview Dana Gardner conducted with Kurt Bittner, a Forrester analyst who focuses on DevOps. I found it so thought provoking that I’d like to share some of its highlights, along with my take on the points it raises.
As I noted above, some people take DevOps as a form of agile development, or an extension of agile practices. Agile development has transformed software development from a lengthy waterfall process to one in which teams collaborate, focus more on code development than specification writing, and business representatives directly participate in the development process. Some people think that implementing agile will result in DevOps, and thereby solve the issue of delivering applications more rapidly.
As Bittner notes, this is unfortunately not correct. First off, many organizational efforts to move to agile development are hampered by poor infrastructure availability; while individual developers can work on their own machines, inevitably there comes a time when a production-like environment must be stood up to evaluate integration and allow more robust testing. This process requires machines to be provisioned, and obtaining them can often take weeks or months. So even agile development qua agile development is challenged when attempted in a traditional infrastructure operations approach.
[Related: Diving into DevOps details]
Beyond this, however, is a bigger issue: even if agile development operates smoothly and can obtain sufficient resources, in all too many IT organizations the development code is handed over to an operations group that continues to execute deployment and management via legacy manual processes. In the end, the overall speed of application delivery is barely changed.
As Bittner observes, it is therefore critical that overall delivery times drop. This requires agility in every part of the application lifecycle, which means every part of the application delivery chain has to up its game. Until every group can execute its tasks quickly, the pace of application delivery will never be fast enough.
Incidentally, while business groups often point to IT as the roadblock in delivering applications and updates fast enough, DevOps can also impose changes on business units themselves. As the excellent book Lean Enterprise: How High Performance Organizations Innovate at Scale points out, delivering changes into the marketplace requires the business side of the house to be agile in terms of developing desired features, responding to IT requirements that are fed back as development groups work on those features, and providing rapid feedback on customer adoption of previously-delivered functionality. It goes without saying that this requires tight coordination and collaboration between business sponsors and IT implementers.
Another “movement” that often accompanies DevOps is a new approach to application functionality and architecture, commonly referred to as microservices. The key concept in this is to decompose the application architecture into multiple smaller execution entities that can be updated independently. By implementing microservices, development groups can avoid the “big bang” release approach, in which many new features are bundled into a single update; since each microservice is on its own release schedule, small updates can be rolled out more frequently. Microservices has only emerged in the last 12 to 18 months, but you can be assured this architectural approach will be a powerful influence on IT groups going forward.
Bittner notes that, absent automation for all IT tasks associated with application development and deployment, DevOps will never be possible. As Bittner puts it:
It’s absolutely essential that, without automation and that data-driven visibility into what's happening in the applications, there's almost no way to deliver these applications at speed. We find that many organizations are releasing quarterly now, not necessarily the same app every quarter, but they have a quarterly release cycle. At quarterly rates of speed, through seat of the pants and sort of brute force, you can manage to get that release out. It’s pretty painful, but you can survive.
If you turn up the clock rate faster than that and try to get down to monthly, those manual processes completely fall apart. We have organizations today that want to be delivering at weekly and daily intervals, especially in SaaS-based environments or cloud-based environments. Those kinds of delivery speeds are inconceivable with any kind of manual processes. As organizations move away from quarterly releases to faster releases, they have to adopt these techniques.
I have witnessed the struggles of IT organizations as they attempt to move more quickly by “brute force.” Pressure and stress build as demands for faster and faster application updates multiply. Unfortunately, all too many IT organizations find themselves “too busy” to invest in automation; for them, the move to DevOps will only occur once it’s unavoidably obvious to everyone that existing processes are unable to meet expectations. The pity is that all too often many employees flee the environment, burnt out on unrealistic demands. Then the organization faces a double whammy: the need to implement automation but finding itself short of the experienced talent that is familiar with existing processes and thereby best suited for automating them.
I’ll have more to say about this issue in the future, but the item that Bittner nails is that automation isn’t a “nice to have,” it’s fundamental to DevOps.
One aspect of the move to DevOps that Bittner discusses is the need to high-level sponsorship. I’m paraphrasing his statement here, but the notion is that end-to-end automation requires work to flow unimpeded across different IT groups; in other words DevOps pierces organizational boundaries. Naturally, most organizations, including IT, resist this. Each group has its established way of doing things and are loath to change.
For this reason, DevOps initiatives that start as grassroots efforts by low-level personnel are unlikely to achieve success beyond isolated experiments. Once the DevOps prototype goes public, various groups will identify reasons that preclude fully implementing cross-organization automation and cite them as insurmountable obstacles in going further.
The main point of this is that DevOps cannot succeed without someone senior enough that all of the various groups report to him or her, and that person visibly sponsors a DevOps investment in terms of time and money.
On the other hand, DevOps cannot succeed without low-level personnel ready and eager to make it work. It’s tempting to view mid-level management as the primary barrier to DevOps success, but I’ve seen plenty of IT personnel who are uninterested in making any changes; they’ve got the current system down pat, and why should they bother with something new
That attitude is not universal however; in every organization there are other people impatient to change things and excited about the opportunity to take part. These are the people who should be engaged in early DevOps implementation efforts, and they should take the lead in rolling it out across the larger organization. Once critical mass is achieved, even the stick-in-the-muds will figure out that the momentum is irresistible and they’ll need to get with the program – or leave.
As it probably clear, I enjoyed Bittner’s interview and found his perspective compelling. Nevertheless, I’d like to identify one caveat I had regarding the interview. In discussing how enterprise IT groups move to DevOps, he notes that it is typically a process of incremental adoption – first comes automated infrastructure provisioning, then continuous integration and automated testing, and finally organizational restructuring to product-oriented teams that contain members of each IT discipline and who collaborate to accelerate overall application delivery. This makes sense – as he results of one phase are internalized, the organization can move on to the next bottleneck in application delivery, and gradually implement a full DevOps solution.
Moving from the first phase to the third, according to Bittner, commonly can take years. And it’s here that I have my caveat. From my point of view, enterprise IT organizations don’t have years. The pressures of the Third Platform are all too present, and in every industry startups and some incumbents are well down the road of leveraging the Third Platform. Embarking on a DevOps journey that is measured in years leaves IT organizations (and the companies of which they are part) dangerously exposed and vulnerable to disruption. This isn’t to criticize Bittner, but merely to note that today’s business climate is not one in which years-long efforts are adequate or acceptable.
Whether you’re part of an IT organization that is well down the DevOps path, or part of one considering how to respond to this new buzzword, I predict you’ll find the impetus behind DevOps unavoidable. Simply stated, the need for application speed is a factor in today’s “software is eating the world” economy, and every IT organization will need to respond. Bittner’s interview is an excellent primer on trends and how to get started, and you owe it to yourself to read the full article.