The most innovative DevOps trends of 2018, and what to look out for in 2019
Seems like you can’t talk tech these days without hearing about DevOps. What is it? How does it work? And what does it mean for your business? Honestly, even the most technical folks can’t seem to agree on a single definition. But for the purposes of our discussion, let’s say it’s a shift in the process from working in silos and “handing work off” from one team to another, to having development staff collaborate with operations staff throughout the development lifecycle of a new service, feature, or update.
DevOps aims to bridge the gap between developers and operations staff to improve efficiency, transparency and time to market.
The upside is greater transparency, teamwork, flexibility, scalability, and faster time to market. And to use the parlance of every middle school cool kid, “everyone’s doing it.” So, should you? Are you? The short answer? YES.
In an effort to get you caught up, and since everyone loves a “Top Five” list – we’ve decided to give you two. First, we’re going to take a look at what we think were the top five DevOps breakthroughs of 2018 – and then, the top five DevOps trends to keep an eye out for in 2019.
Our criteria for these 10 is simple – they’re all moving the idea of DevOps forward, and have either already made a huge impact on DevOps, or have the potential to. Let’s get started by looking back at 2018 DevOps developments, in no particular order:
- Time Series Databases (TSDBs): Used to optimize and store data points indexed in time order, such as server metrics or service/application performance, a TSDB helps measure change over time. By capturing detailed production system metrics, TSDBs enable one of DevOps’ key components — gaining observability into how production systems are performing. This empowers both the decision-making and collaboration processes, making TSDBs a “go-to” in the DevOps toolbox.Need more reason to believe? The developer-focused industry analyst firm RedMonk published The State of the Time Series Database Market in 2018, which lists several commercial and open-source time series databases and their technical challenges and use cases. Time series databases even have their own dedicated Wikipedia page, signaling that they have established themselves through independent reliable sources in the media.
- Ubiquitous Monitoring: Originally developed for SoundCloud, in 2018 Prometheus became the go-to DevOps monitoring tool. This reliable, open-source multi-dimensional data model system is based based on Google’s Borgmon previous generation monitoring system (they currently use Monarch internally). It allows organizations to easily monitor multiple microservices within their architecture; including flexible queries and modern alerts as they move through the development lifecycle. This saves engineering teams huge amounts of time and effort across the board. In part due to its flexibility and simplicity, the system has been widely accepted and adopted by developers and users, and is now recognized as a critical component of any DevOps implementation.
- The Big Boys are Playing: We also know that DevOps is reaching critical mass because of the number of large organizations making strategic moves towards it. Besides the growing number of high-profile DevOps conferences popping up in the past 12 months (see below), we’re also talking moves by Microsoft, Cisco, and IBM. In fact, DevOps Research and Assessment (DORA) was recently acquired by Google Cloud, and “DevOps Engineer” is one of the most sought-after job titles of 2018, according to a LinkedIn report. So, we can safely say that DevOps isn’t a fad.
- Experts Are Finally On Board: After a few years of hand-wringing, a simple Google search will show you that even the most seasoned tech experts are telling companies that DevOps is the way to go. The DevOps for Dummies book, published by IBM, is already on its 3rd edition and makes a strong case for the business need and value of DevOps.
- DevOpsDays are Everywhere: At last count, there are almost 50 DevOpsDays conferences scheduled for 2019. Organized locally and guided by core organizers globally, DevOpsDays are a worldwide series of technical conferences that make it easy to learn about the latest DevOps trends and approaches. In the past decade, DevOpsDays has grown from a single event to a global presence. As a matter of fact, there’s probably one scheduled near you.
So, 2018 did a great job setting the stage. DevOps is here to stay, and it’s definitely not sitting still. Where will DevOps go in 2019? Here are the trends and innovations we’re looking out for. Hint? They’re all happening behind the curtain:
- Metrics Standards: As more organizations jump on board the DevOps train, what we’ll see more of in 2019 are accepted and adopted standards for metrics, from deployment frequency and time to test pass percentage and availability. For example, OpenCensus provides vendor-agnostic libraries for both metrics and tracing.While OpenCensus is still in the formative stages of development, the approach of portable standards for metrics driven by not just corporate sponsors but industry experts is one that we expect to evolve and shape the needs of tomorrow’s systems which will handle trillions of metrics.
- Tracing Standards: Similar to above, with OpenTracing we’ll also see vendor-neutral APIs and instrumentation for distributed tracing libraries. Along with Jaeger (founded by Uber), that means even more tracing visibility into distributed applications. These rapidly developing tracing standards will also help bring the industry together and drive DevOps forward. The availability of cloud-based distributed tracing such as AWS X-Ray has shown that distributed tracing is also here to stay.
- Log Aggregation: Scalable, flexible on-prem logging addresses a key challenge in the collaboration between development and operations. One space leader with a solution optimized for users running Prometheus and Kubernetes is Loki by Grafana. This logging backend is “…optimized to search, visualize and explore your logs natively in Grafana,” and is leading the way in pattern recognition. Expect to see more solutions that follow this approach lead in 2019.
- Low-Level Systems Observability: With a microscopic view into Linux low-level kernel and system code at runtime, Extended Berkeley Packet Filter (eBPF) allows BPF to go beyond simply filtering packets, enabling custom analysis programs to be executed on Linux dynamic tracing, static tracing, and profiling events.The ability to introspect low-level systems details at large scale will become critical for large infrastructures, where small gains on individual nodes translate to large efficiencies across availability zones.
- On-Premise Enterprise Software: With all of the momentum that cloud-based services have gained over the past several years, expect the pendulum to start swinging back towards on-premise solutions — especially with the rising number of cloud breaches we’ve seen in that same time period.The need for scalable, cost-effective solutions for logging, container orchestration, and other infrastructure components such as AWS Outposts has been compounded by the success of easy to adopt hosted web services that have proven themselves at a smaller scale. The cloud spurs innovation and adoption, but when solutions need to grow up into mission-critical roles in large enterprises, on-premise installations are what is needed for large scale.
So, moving forward, we can expect to see more developments in the infrastructure that makes DevOps work, especially in terms of transparency, access, and visibility. This “behind the curtain” growth is just as important as the development of DevOps processes themselves, as they enable further implementation. This also means giving decision-makers not only more in-depth information but more valuable, actionable insight.
DevOps will continue to mature, and 2019 will be a big year in terms of getting more organizations on board, more consolidation, more experience, and more data and insight to build best practices. The true test, as always, will be the actual gains in efficiency, transparency, and time to market.
