AWS Cloudwatch Support

This month we pushed native Cloudwatch support – any metric that you have in Cloudwatch can now be added to graphs and dashboards, and alerts can be created for them.

Auto Scaling (AWS/AutoScaling)
AWS Billing (AWS/Billing)
Amazon DynamoDB (AWS/DynamoDB)
Amazon ElastiCache (AWS/ElastiCache)
Amazon Elastic Block Store (AWS/EBS)
Amazon Elastic Compute Cloud (AWS/EC2)
Elastic Load Balancing (AWS/ELB)
Amazon Elastic MapReduce (AWS/ElasticMapReduce)
AWS OpsWorks (AWS/OpsWorks)
Amazon Redshift (AWS/Redshift)
Amazon Relational Database Service (AWS/RDS)
Amazon Route 53 (AWS/Route53)
Amazon Simple Notification Service (AWS/SNS)
Amazon Simple Queue Service (AWS/SQS)
AWS Storage Gateway (AWS/StorageGateway)


This check monitors your Amazon Web Services (AWS) resources and the applications you run on AWS in real-time. You can use the CloudWatch Check to collect and track metrics, which are the variables you want to measure for your resources and applications.

From the CloudWatch Check, you can set alerts within Circonus to send notifications, allowing you to make changes to the resources within AWS.  For example, you can monitor the CPU usage and disk reads and writes of your Amazon Elastic Compute Cloud (Amazon EC2) instances and then use this data to determine whether you should launch additional instances to handle increased load. You can also use this data to stop under-utilized instances to save money. With the CloudWatch Check, you gain system-wide visibility into resource utilization, application performance, and operational health.

Circonus takes the AWS Region, API Key, and API Secret, then polls the endpoint (AWS) for a list of all available Namespaces, Metrics, and Dimensions that are specific to the user (AWS Region, API Key, and API Secret combination). Only those returned are displayed in the fields. The names that are displayed under each Dimension type (for example: Volume for EBS) are all instances running this Dimension type and have detailed monitoring enabled.

For information on the master list of Namespace, Metric, and Dimension names available and additional information on Cloudwatch in general, see AWS’s Cloudwatch documentation.

JSON Over HTTP – Data Collection Made Simple

At Circonus, one of our goals is to try to make it as easy as possible to monitor your data. One of the ways we do this is to allow data formatted in JSON to be pushed or pulled over HTTP into Circonus. Since HTTP is spoken everywhere, and JSON is understood everywhere, this allows for easy metric submission so you can collect, store, graph, and analyze everything that you care about.

The HTTPTrap check type accepts JSON payloads via HTTP PUT requests. This allows you to push data from your devices or applications directly into Circonus. This is useful for data that happens sporadically, instead of at a regular or constant interval. HTTPTraps also let you send histogram data into Circonus, so you can see the whole picture instead of one aspect of your data.

The JSON check type gets data from an HTTP endpoint at the interval you select. This allows you to make applications that expose metrics in a JSON format that can be polled regularly from Circonus. These checks allow you to specify a username/password, port, and any additional headers, which gives you security and flexibility in what you allow to connect to your hosts.

One of the major shortcomings with JSON in most languages is the ability to deal with large numbers. Our parser works around that by allowing you to send the number as a string. This means there is no data that you’re interested in that we can’t collect or accept.

The ability to use JSON as a format for data also allows you to write your own data collector. For instance, Gollector was written by the folks at Triggit who wanted to have an agent that relied on the proc filesystem and C POSIX calls. Additionally, both Panoptimon (written in Ruby) and our very own nad agent (written in Node.js) utilize JSON to send system information. Customized agents like these allow you to adapt Circonus to your infrastructure and monitoring needs.

To show just how easy it is to format data so Circonus can read it, this is an example Python script that runs once per minute to generate some randomized data. Once you create an HTTPTrap check in Circonus, you can look at the check to get the URL that should be used in the PUT call. The example includes submitting strings, small numbers, large numbers, and a set of numbers that can be used for histogram data. Similar setups can be used in other languages and in your own custom applications.

import json
import urllib2
import time
import random

# Use the URL provided in the UI from the Circonus HTTPTrap check
httptrapurl = ""

    # Make up the data
    data = {
            "number": random.uniform(1.0, 2.0),
            "test": "a text string",
            "bignum_as_string": "281474976710656", 
            "container": { "key1": random.randint(1200, 1300) },
            "array": [
                random.randint(1200, 1300),
                { "crazy": "like a fox" }
            "testingtypedict": { "_type": "L", "_value": "12398234" },
            # Set the type to "n" for histogram-enabled data
            "histogramdata": { "_type": "n", "_value": [int(1000*random.betavariate(1,3)) for i in xrange(10000)] }
    jsondata = json.dumps(data)

    # Form the PUT request
    requestHeaders = {"Accept": "application/json"}
    req = urllib2.Request(httptrapurl, jsondata, headers = requestHeaders)
    req.get_method = lambda: 'PUT'
    opener = urllib2.urlopen(req)
    putresponse = json.loads(

    # Print the data we get back to the screen so we can make sure it's working
    print putresponse
    print jsondata

    # Wait a minute

This will show up in Circonus as:

You can refer to the Circonus User Manual for more details about the HTTPtrap check. Also, please refer to the information there to import our certificate if you see the following error while following these instructions:
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>