Exploring Keynote XML Data Pulse

I’ll be the first to admit that the Circonus service can be somewhat intimidating. Sometimes it is hard to puzzle out what we do and what we don’t do. Case in point: perspective-based transactional web monitoring.

Many people have asked us, given our global infrastructure, why don’t we support complex web flows from all of our global nodes and report back granular telemetry regarding the web pages, assets and interactivity. The short and simple answer is: someone else is better at it. It turns out they are a lot better at it.

Keynote has been providing synthetic web transaction monitoring via their global network of nodes for many years and have an evolved and widely adopted product offering. So, why all this talk about Keynote?

But why?

You might ask why it is important to get deep, global data about web page performance into Circonus. It’s already in Keynote, right? Their tools even support exploring that data and arguably better than within Circonus?

The reason is simple… you’re other critical performance data is in Circonus too. Real-time correlations, visualization and trending can’t happy easily unless the data is available in the same toolset. Web performance is delivered by web infrastructure. Web performance powers business. Once all your performance data is in Circonus, you can can tie these three macro-systems together in a cohesive view and produce actionable information quickly.

The story of how we made this possible is, as most good stories are, rife with failures.

Phase Failure: the Keynote API

For over a year, we’ve had support for extracting telemetry data from Keynote via their traditional API. For over a year, most of our customers had no idea… because it was in hidden beta. It was hidden because we struggled to make it work. Honestly, the integration was painful due to the API allowing us to pull only a single telemetry point at a time. It was so painful that we struggled to add any real value on top of the data they stored. The API is so bad (for our needs) it almost looks like Amazon Cloudwatch (a pit of hell deserving of a separate blog post).

If you look at a standard deployment of Keynote, you might find yourself pulling data 200-300 measurements from 15 different locations every minute. For Circonus to pull that feed, we’d have to do 4500 API calls/minute to Keynote for each customer! That’s not good for anyone involved.

Phase Success: the Keynote XML Data Pulse

Recently, our friends over at Keynote let us in on their new XML Data Pulse service which looks at their data more “east and west” as opposed to “north and south.” This newer interface with Keynote’s global infrastructure allows us to pull wide swaths of telemetry data into our systems in near real-time… just like Circonus wants it.

If you’re a Keynote customer and are interesting in leveraging our new Data Pulse integration, please reach out to your Keynote contact and get setup with a Data Pulse agreement.