Technology has a long and storied tradition of moving quickly and changing often. So much so, that it can appear as though no idea, methodology, or framework will last more than a few CPU cycles in the industry. So, it may come as something of a surprise to learn that DevOps (the vaunted closing of the gap between development and operations) is nearly 11 years old (it was officially introduced to the world at DevOps Day 2009 in Belgium). Even more surprising is how slowly the DevOps concept has matured since then. This blog will provide some foundational knowledge about the concept and try to dispel some myths that have plagued its adoption over the course of its existence.
What DevOps is NOT
First and foremost, it is important to understand the things that DevOps is not before diving into the theory and promise of what DevOps can be. DevOps is not a methodology, nor is it a technology. No one sells “DevOps”. There are plenty of tools in the marketplace touting how compliant they are with DevOps, which is essentially like telling people that a tool is compliant with ITIL. It is not a standard with which an organization can comply. In fact, many would argue that DevOps is not a framework with which one can align, either. This is one of the reasons that the concepts introduced by DevOps are difficult to adopt and even harder to define at the organizational level.
Convincing leadership that something this abstract can add value, and then promptly asking to reorganize critical pieces of the organization in order to adopt these practices is a very tall order. And many, if not most, companies balk at the notion—at least in part. This only adds to the frustration of those teams trying to adopt DevOps. And for those companies that refuse to fully embrace the practices and truly buy into the notion that development and operations can, in fact, live more harmoniously and produce better results faster, it’s easy to point at the pieces they do adopt and blame the entire concept for any failure that may occur.
What is DevOps?
So, enough about all the possible vectors of abject self-destruction. What is DevOps? And more importantly, why should anyone care? First, DevOps is a set of practices and values that an organization adopts in order to close the gap between ideation of products and services, and the ultimate maintenance and support of those same products and services. Simply put, DevOps provides a holistic environment where the entire value chain is considered and improved through automation, collaboration, and continuous feedback.
That is a pretty lofty goal. And its roots can be traced to Agile methodologies from the early 2000s, Lean methods in technology, and on and on back to the original quality and efficiency models designed by visionaries like Deming and Imai in the manufacturing world. At its core, DevOps aims to take the best of all of these and create a new culture within our technology industry which produces better products and services and then supports them in an end-to-end fashion, garnering feedback at every step and, more importantly, in a continuous fashion.
This requires a great deal of automation in testing, release, and even operations. And this is where a lot of people start to lose faith. In fact, if one were to read some of my earlier thoughts on this concept, one might learn that this was one of my biggest concerns in adoption. How, in the name of all that is good and effective in IT, could testing be automated? I spent years in operations dealing with weekends on-call handling “fully tested” code and the fallout it caused on numerous systems. I had little respect for the manual processes used to test release packages, which led me to believe that automation would only multiply my gray hairs. And, as a process nerd, there is a lot of evidence to support this. Automating a bad process only makes it faster and more consistently bad. Additionally, it seemed ludicrous that automated testing could possibly take the place of high quality, human intervention. Then it was explained in a way that made perfect sense: humans are fallible and have biases that can lead to both lazy and erroneous testing methods. It wasn’t the testing itself that was the problem. It was the testing execution that was causing all those sleepless nights.
DevOps focuses on creating a repeatable test process that can be fully automated and inserted into the overall development lifecycle. This accomplishes both the goal of eliminating human bias as well as shortening the timeline between ideation and delivery of a product. This, not so surprisingly, supports the traditional approach to any process in the IT service management ecosystem: optimize the process, then automate it to increase efficiency. It was a huge hurdle for me, mentally, to finally realize the value in this. But once that obstacle was removed, I got interested in what other things this crazy notion of DevOps might be capable of creating.
Then Came the CALMS Model
Shortly after the introduction of DevOps, a model was produced by Damon Edwards and John Willis in order to help explain why DevOps is important. In 2010, these two authors established what came to be known as the CAMS model. It stood for Culture, Automation, Measurement, and Sharing. A couple of years later, Lean was added to the acronym, expanding it to CALMS. These five values make up the core of what DevOps can do for an organization. They are sometimes referred to as the “why” of DevOps.
Culture
Culture is at the heart of DevOps. DevOps is first and foremost, a movement to change the way technology professionals work and transform organizations—or, as it is sometimes said, DevOps solves a human problem. Adoption of more collaborative processes, the elimination of siloed workforces, and the establishment of a feedback driven environment all mean a major shift in thinking and values for the entire organization. Thus, without this first key ingredient, and the buy-in at all levels to accomplish the shift, the rest has a pretty high chance of failure.
In order to understand how culture needs to change, organizations can examine how others have been successful with similar challenges and opportunities; however, it is important to remember that each organization has a unique culture that must be transformed in its own way (and its own time). There is no “magic button” that one can press to suddenly establish a successful DevOps oriented culture.
Automation
Automation, as previously stated, is also a crucial part of a successful DevOps transformation. Traditionally, IT has simply thrown more technology (or more bodies) at a problem hoping that some new resource might be the cornerstone that props up the rest of the structure. Unfortunately, we also discovered that this gets very expensive very quickly, and most businesses (surprise!) don’t like to throw good money after bad. As a result, organizations began to cut budgets and outsource technology to save money in the hopes that either their IT folks could do more with less (always a winning combination) or that an outside “expert” organization would cure all their ills. Neither of these approaches proved particularly successful most of the time. But, if we look to optimize those processes we already have in place, such as testing and release of new features and applications, then automate them as much as possible, we accomplish two elusive goals: greater efficiency with fewer resources, and higher quality products and services.
Lean
Lean is a term that is tossed about quite a bit, especially within software development and project management. What is at the root of this value is the need to eliminate waste. All products and services come with core functions that are valuable to customers; however, nearly every product or service has enhancing features that are valuable to only a few, if any, customers. The Lean value seeks to ensure that release of a product or service is not delayed due to the inclusion of such features. We often hear the term ‘minimum viable product’ in Agile, and that is what Lean really means. Ensure that the features and components that are absolutely necessary for the current iteration are complete and functional, but do not spend a lot of time trying to create the perfect product.
Measurement
Measurement speaks to the idea of continual improvement. This concept has been around about as long as industry itself. DevOps primarily adopted Kaizen to establish a method for continual improvement, and Lean and Deming’s Plan-Do-Study-Act methods are also commonly employed. This model was pioneered by Masaki Imai in the mid-twentieth century. It translates to “Change for Better”, making it rather appropriate for adopting best practices for continual improvement. Measurement is a key component for improvement. As the saying goes, “You cannot improve what you do not measure.” So it is with DevOps. You must establish a baseline for performance, and then continuously re-evaluate progress. Along with models for improvement, there are myriad tools used to measure processes, and as automation picks up, it becomes important to leverage technology to measure our progress.
Sharing
Finally, there is Sharing. Collaborative working has been a foundational element of most advances in technology over the past decade, so it is no surprise that the DevOps movement places great emphasis on this idea as well. Where DevOps has excelled, however, is in the concept of including collaboration across the entire value chain. Rather than simply having developers collaborating prior to release, then handing off a finished product to operations to administer and maintain, DevOps strives to break down that barrier. Bringing developers closer to operations, requiring a level of responsibility traditionally rejected by both developers and operators, this movement places both greater burden, but also greater understanding of a product or service from end to end. And this is something that nearly every framework, methodology, and standard has tried for half a century to capture. The idea of continuous feedback loops for every product and/or service IT provides, rapid response to bugs and incidents, and responsibility across the entire value stream all coalesce to create a more customer-focused, effective, and efficient organization.
Truly Embracing DevOps
There are many more concepts and practices within the DevOps movement, and each has a part to play. It is important to recognize that each organization must choose its own path for adopting these and adapting them to its particular needs. But it is critical to truly embrace the concepts and gain the buy-in at all levels. Each of the five values presented in the CALMS model can be used when building the case for DevOps. When used in concert, this model can produce lower resource costs, greater customer satisfaction, and an overall more cohesive workforce.