One-Minute-Resolution Data Storage is Here!

Have you noticed your graphs looking better? Zoom in. Notice more detail? That’s what our switch from 5 to 1-minute-resolution data storage has done for our SaaS customers. Couple that with our 7 years of data-retention and you’ve got an unrivaled monitoring and analytics tool. Dear customers, your computations are becoming more accurate as we speak!

“Keeping 5 times the data, combined with visualizing data in 1 second resolution in real-time, gives us unprecedented ability to forecast future trends and ask questions like ‘what happened last year during that big event’,” adds Circonus CEO, Theo Schlossnagle.

The switch from 5 to 1-minute-resolution data storage means more data points.

One Minute Data Storage Now and Then
One Minute Data Storage Then and Now

Companies around the world depend on Circonus to provide unparalleled insight into all aspects of their infrastructure. Their websites typically have large swings in traffic due to a variety of factors, such as product launches, news events, or holiday shopping and other annual events. Our switch to 1-minute-resolution data storage is great news for our SaaS customers.

“Our web infrastructure is essential to our business,” says Kevin Way, Director of Engineering Operations at Monetate. “Circonus gives us the insight we need to provide our customers with the reliability they deserve. The ability to use detailed data from previous events to predict a future event is incredibly valuable! This is a major differentiater for Circonus.”

“At Wanelo we invested heavily into metrics and visibility into our infrastructure, and Circonus is a huge part of our strategy. Being able to visualize our data in 1-minute resolution gives us unprecedented ability to diagnose and remediate issues across all aspects of our infrastructure,” says Konstantin Gredeskoul, CTO at Wanelo.com.

Along with the adoption of Devops, companies are increasingly dependent on highly dynamic cloud environments, provisioning and deprovisioning infrastructure as needed. This affects decisions made at every level of an organization, from the CEO, to the product team, to IT Operations. A detailed history of usage and traffic patterns – such as from AWS, Azure, Google Cloud, Heroku, Rackspace, or one’s own private cloud infrastructure – gives an organization immeasurable insight into network performance monitoring, as well as the costs associated with providing customers with a world-class experience.

On the Move: Circonus Mobile

It’s been a long time coming, but it’s finally here: a Circonus that’s optimized for low-resolution devices like your smartphone or tablet! For the past few weeks, eagle-eyed Circonus customers may have noticed links to the mobile site creeping into various places, such as the login page (login.circonus.com) and the application footer. Although we officially “flipped the switch” a few weeks ago to enable the new Circonus Mobile experience, it is definitely a work in progress. To visit Circonus Mobile, visit m.circonus.com or use the “Switch to Mobile Site” link in the main site footer.

The most common need when using Circonus on a smartphone is being able to easily and quickly respond to alerts, so we made alerting functionality a priority when developing this new interface. The Current Alerts and Alert History pages are complete, already having feature parity with the main site. Also fully implemented are the Account Profile and Account Brokers pages, along with the User Profile and User Preferences pages, but most other pages are being implemented in a phased manner, beginning with read-only versions. TIP: when viewing a paginated list of items in Circonus Mobile, you may swipe left and right to page through the list (if you’re using a browser which supports touch events).

Our long-term goal for Circonus Mobile is not only achieving feature parity with the main site, but going further with push notifications and other native app functionality. But to give you a peek into our short-term goals, we’re currently working on making the Maintenance page fully functional, and after that we will move on to read-only versions of the Graphs and Worksheets pages.

If you use Circonus Mobile and don’t see a feature from the main site that you want, please let us know! We’re continually implementing features on the mobile site, but we’re more than willing to rearrange our implementation roadmap based on our customers’ needs.

Interacting with Circonus through your text editor – circonusvi

I’m a big fan of command line tools for many tasks. There are just some tasks that can be done quicker and easier with a little typing rather than pointing and clicking. A little while ago, I discovered a gem of of a command line tool called ldapvi. Without going into too much detail, this tool lets you run a query against an LDAP server where the results show up in your favorite text editor. The magic part is that when you edit that file and save it, whatever changes you made are then pushed back to the server. This method of working can be extremely flexible, as you have all of the power of your text editor at your fingertips to make both small and sweeping changes quickly and easily.

When the new Circonus API was released, I realized that it was now possible to do the same thing for Circonus with relative ease. It would have been impossible to do with the old API, but the new one was designed with consistency in mind, and a simple (dare I say the word REST) interface. The result is circonusvi. Circonusvi essentially allows you to interact directly with the circonus API via your text editor, and it works as follows:

  • You decide what you want to work with: check bundles, rule sets, worksheets and so on, and pick that as your endpoint.
  • Circonusvi queries the API and lists all resources, which are returned as JSON.
  • Optionally, circonusvi filters this output to only show the resources you want.
  • This JSON (slightly modified) is shown in your text editor, where you can make changes (edit checks, remove checks, add new checks) as needed.
  • Circonusvi identifies what changes were made, and pushes them back to the server.

Here’s a simple example, taken from an actual situation we came across: needing to switch contact groups for a series of rules. We had some rules which were a severity 2 (our convention is that sev2 alerts are mail only), and we realized that we really should have them be sev1 (in other words, wake up the on call SA). To make this change in the web interface, you need to go into each rule individually, change the criteria from sev2 to sev1, and change the contact group associated with sev1 alerts. This is time consuming and error prone, but at the same time, it’s a one-off task that you probably don’t want to write a script for.

First, we identify the contact groups in circonusvi:

./circonusvi.py -a example_com -e contact_group

{
    # Mail only
    "/contact_group/123": {
        "aggregation_window": 300,
        "contacts": {
            "external": [
                {
                    "contact_info": "sa@example.com",
                    "method": "email"
                }
            ],
            "users": []
        },
        "name": "Mail only",
        "reminders": [
            0,
            0,
            0,
            0,
            0
        ]
    },
    # On call
    "/contact_group/124": {
        "aggregation_window": 300,
        "contacts": {
            "external": [
                {
                    "contact_info": "oncall@example.com",
                    "method": "email"
                }
            ],
            "users": []
        },
        "name": "On call",
        "reminders": [
            0,
            0,
            0,
            0,
            0
        ]
    }
}

This shows two contact groups, with the ID of the mail only contact group being /contact_group/123, and the on call contact group being /contact_group/124.

Next, we need to do the actual editing:

./circonusvi.py -a example_com -e rule_set 'metric_name=^foo$'

Here we specified a filter on the rules to match only those rules that apply to a metric named ‘foo’. The pert before the equals sign is which key you filter on. You can specify any key in the results for the filter, as long as it has a simple value (i.e. string or number). The right hand side of the filter is a regular expression. Any entry that matches the regular expression will be displayed. A small section of the results are shown below:

{
    "/rule_set/1234_foo": {
        "check": "/check/1234",
        "contact_groups": {
            "1": [],
            "2": [
                # Mail only
                "/contact_group/123"
            ],
            "3": [],
            "4": [],
            "5": []
        },
        "derive": "counter",
        "link": null,
        "metric_name": "foo",
        "metric_type": "numeric",
        "notes": null,
        "parent": null,
        "rules": [
            {
                "criteria": "on absence",
                "severity": "2",
                "value": "t",
                "wait": 10
            }
        ]
    },
    ...

There are two things we want to change here. First the severity when the rule fires. This is a simple search/replace in your text editor. I’m a vim fan, so:

:%s/"severity": "2"/"severity": "1"/

Next, we need to make sure that severity 1 alerts go to the on call group. They don’t currently:

:%s/"1": [],/"1": [ "/contact_group/124" ],/

If you want, you can remove the sev2 contact group by deleting that section:

:g/"/contact_group/123"/d

The ‘comment’ line above can be left alone, or deleted too with a similar command.

Once you are finished, your file will look something like this:

{
    "/rule_set/1234_foo": {
        "check": "/check/1234",
        "contact_groups": {
            "1": [ "/contact_group/124" ],
            "2": [
                # Mail only
            ],
            "3": [],
            "4": [],
            "5": []
        },
        "derive": "counter",
        "link": null,
        "metric_name": "foo",
        "metric_type": "numeric",
        "notes": null,
        "parent": null,
        "rules": [
            {
                "criteria": "on absence",
                "severity": "1",
                "value": "t",
                "wait": 10
            }
        ]
    },
    ...

Save and quit, and you’ll be presented with a prompt asking you to confirm your changes, and give you a chance to preview the changes before submitting them:

0 additions, 0 deletions, 85 edits
Do you want to proceed? (YyNnEeSs?)

I would recommend previewing exactly what has changed before submitting. Pressing S will show a unified diff of what changed. Once you proceed, circonusvi will apply the changes via the API, and you’re all done!

This method of interacting with the API is most useful when making edits or deletions in bulk (if you delete an entry in the text editor, circonusvi will delete from the API). Additions are possible also, but are more cumbersome as you have to create the JSON precisely from scratch. For bulk additions, there are other scripts available such as the
circus scripts. However, using your editor to make bulk changes is a very powerful way of interacting with an application. Give it a try and see how well it works for you.

Mark Harrison leads the Site Reliability Engineering team at OmniTI where he manages people and networks, systems and complex architectures for some of today’s largest Internet properties.