Video: Architecture of a Distributed Analytics/Storage Engine for Massive Time-Series Data

The numerical analysis of time-series data isn’t new. The scale of today’s problems is. With millions of concurrent data streams, some of which run at 1MM samples per second, the challenge of storing the data and making it continuously available for analysis is a daunting challenge.

At Circonus, we designed such a solution. Our CEO, Theo Schlossnagle, during Applicative 2015, discusses the approach and the technical details of how the system was constructed. Check out his talk…

One-Minute-Resolution Data Storage is Here!

Have you noticed your graphs looking better? Zoom in. Notice more detail? That’s what our switch from 5 to 1-minute-resolution data storage has done for our SaaS customers. Couple that with our 7 years of data-retention and you’ve got an unrivaled monitoring and analytics tool. Dear customers, your computations are becoming more accurate as we speak!

“Keeping 5 times the data, combined with visualizing data in 1 second resolution in real-time, gives us unprecedented ability to forecast future trends and ask questions like ‘what happened last year during that big event’,” adds Circonus CEO, Theo Schlossnagle.

The switch from 5 to 1-minute-resolution data storage means more data points.

One Minute Data Storage Now and Then
One Minute Data Storage Then and Now

Companies around the world depend on Circonus to provide unparalleled insight into all aspects of their infrastructure. Their websites typically have large swings in traffic due to a variety of factors, such as product launches, news events, or holiday shopping and other annual events. Our switch to 1-minute-resolution data storage is great news for our SaaS customers.

“Our web infrastructure is essential to our business,” says Kevin Way, Director of Engineering Operations at Monetate. “Circonus gives us the insight we need to provide our customers with the reliability they deserve. The ability to use detailed data from previous events to predict a future event is incredibly valuable! This is a major differentiater for Circonus.”

“At Wanelo we invested heavily into metrics and visibility into our infrastructure, and Circonus is a huge part of our strategy. Being able to visualize our data in 1-minute resolution gives us unprecedented ability to diagnose and remediate issues across all aspects of our infrastructure,” says Konstantin Gredeskoul, CTO at

Along with the adoption of Devops, companies are increasingly dependent on highly dynamic cloud environments, provisioning and deprovisioning infrastructure as needed. This affects decisions made at every level of an organization, from the CEO, to the product team, to IT Operations. A detailed history of usage and traffic patterns – such as from AWS, Azure, Google Cloud, Heroku, Rackspace, or one’s own private cloud infrastructure – gives an organization immeasurable insight into network performance monitoring, as well as the costs associated with providing customers with a world-class experience.

Video: Math in Big Systems

Every year the esteemed Usenix organization holds their LISA conference. LISA has transformed slowly over the years as systems, architectures, and the nature of large-scale deployments have changed, but this year represented the largest change to date.

“The format of the conference was substantially different and I believe it (changed) for the best. The topics, content, and speakers were both relevant and fantastic while keeping just enough of the UNIX neckbeard vibe to make it familiar.” – Theo Schlossnagle

his year at LISA, our CEO presented what we’ve been doing in the realm of automatic anomaly detection on high-frequency time-series data; an otherwise dry subject was cordially delivered by Theo and very well received.

LISA14: Math in Big Systems by Theo Schlossnagle