Histograms
The Circonus Platform leverages OpenHistogram, an open source data structure to deliver real-time, high frequency telemetry intelligence and insights.
Histograms are a data structure for representing large quantities of samples (such as the latencies of all API requests). OpenHistogram is a vendor-neutral log-linear histogram technology for the compression and analysis of telemetry data. OpenHistogram efficiently stores an unlimited volume of high-frequency data for years using sophisticated data compression, enabling users to see a more complete and real-time picture of their underlying systems.
“We use Circonus’ histograms extensively to surface issues before users notice them. We also have alerting set up such that if we start to burn error budget in latency for a particular service too quickly, we alert the team. The histograms enable us to identify these issues and make adjustments right away.”
Collect and store ALL telemetry at low cost without sacrificing performance
- Ingest an effectively unlimited volume of metrics at extremely high frequency (trillions of measurements per second) to ensure accurate real-time and historical analysis of metrics.
- OpenHistogram can compress and store ALL data points (trillions), so you can go back to your data at any time to answer any question.
- Eliminate the need for sampling associated with high-volume, real-time telemetry emitted by modern applications. Data sampling was historically performed to conserve computing or networking resources, but no tradeoff is needed with OpenHistogram.
- Reduce costs associated with network egress and metric storage with OpenHistogram’s sophisticated metric compression.
Empower SREs to dynamically create SLOs and error-budgets by calculating aggregations on the fly
- Calculate more advanced aggregates such as arbitrary quantiles, percentiles, inverse quantiles, inverse percentiles, and other distribution characteristics – all without needing to manage and store additional metrics.
- Accurately analyze your SLOs to measure latencies and compliance with KPIs. OpenHistogram enables storage of the complete distribution of data, rather than storing a handful of quantiles.
- Dynamically calculate aggregations on the fly using the complete distribution of your metric data, avoiding the need to pre-calculate aggregations. Redefine SLOs as needed to keep up with the ever evolving needs of your team and business.
- Enable various teams to set their own SLOs rather than be forced to share the same pre-calculated aggregations that only a subset of the team can use.
- Surface issues before users notice them. Alert on real-time SLO compliance issues, or if you are burning through your error budgets too quickly.
Visualize the full distribution of your data to improve performance and reduce latencies
- Visualize real-time data with OpenHistogram. Assess probability distributions, find disruptions and concentrations, discover extreme system loads or drop-offs, and derive new insights — all in real time.
- Visualize historical data. Identify changes in service performance through aggregations, or compare latencies from one software release to another. Use heatmaps to represent histogram data in a color-coded matrix to better visualize usage patterns of a system measured over a period of time.
- Leverage the Circonus Analytics Query Language (CAQL) to create customized and powerful queries against metric data stored as OpenHistogram. Use CAQL to power Circonus or Grafana-based dashboards.
Optimize accuracy and performance
- OpenHistogram’s technique for determining optimal histogram bin sizes has been independently tested and evaluated over the years and consistently deemed superior to other approaches in terms of balancing performance, accuracy, correctness, and usability.
- OpenHistogram provides a mix of storage efficiency and statistical accuracy. Worst-case errors at single digit sample sizes are below 5%, significantly better than the hundreds of percent seen by poor binning approaches and averaging precalculated percentiles, common practices from other vendors.
- OpenHistogram is in Base 10, which eases usability, and it does not require floating point arithmetic, which allows for deployment on embedded IoT devices where floating point may be unavailable for too costly.
Trusted by industry leaders worldwide
