At Circonus, one of our goals is to try to make it as easy as possible to monitor your data. One of the ways we do this is to allow data formatted in JSON to be pushed or pulled over HTTP into Circonus. Since HTTP is spoken everywhere, and JSON is understood everywhere, this allows for easy metric submission so you can collect, store, graph, and analyze everything that you care about.
The HTTPTrap check type accepts JSON payloads via HTTP PUT requests. This allows you to push data from your devices or applications directly into Circonus. This is useful for data that happens sporadically, instead of at a regular or constant interval. HTTPTraps also let you send histogram data into Circonus, so you can see the whole picture instead of one aspect of your data.
The JSON check type gets data from an HTTP endpoint at the interval you select. This allows you to make applications that expose metrics in a JSON format that can be polled regularly from Circonus. These checks allow you to specify a username/password, port, and any additional headers, which gives you security and flexibility in what you allow to connect to your hosts.
One of the major shortcomings with JSON in most languages is the ability to deal with large numbers. Our parser works around that by allowing you to send the number as a string. This means there is no data that you’re interested in that we can’t collect or accept.
The ability to use JSON as a format for data also allows you to write your own data collector. For instance, Gollector was written by the folks at Triggit who wanted to have an agent that relied on the proc filesystem and C POSIX calls. Additionally, both Panoptimon (written in Ruby) and our very own nad agent (written in Node.js) utilize JSON to send system information. Customized agents like these allow you to adapt Circonus to your infrastructure and monitoring needs.
To show just how easy it is to format data so Circonus can read it, this is an example Python script that runs once per minute to generate some randomized data. Once you create an HTTPTrap check in Circonus, you can look at the check to get the URL that should be used in the PUT call. The example includes submitting strings, small numbers, large numbers, and a set of numbers that can be used for histogram data. Similar setups can be used in other languages and in your own custom applications.
import json
import urllib2
import time
import random
# Use the URL provided in the UI from the Circonus HTTPTrap check
httptrapurl = "https://trap.noit.circonus.net/module/httptrap/01234567-89ab-cdef-0123-456789abcdef/mys3cr3t"
while(1):
# Make up the data
data = {
"number": random.uniform(1.0, 2.0),
"test": "a text string",
"bignum_as_string": "281474976710656",
"container": { "key1": random.randint(1200, 1300) },
"array": [
random.randint(1200, 1300),
"string",
{ "crazy": "like a fox" }
],
"testingtypedict": { "_type": "L", "_value": "12398234" },
# Set the type to "n" for histogram-enabled data
"histogramdata": { "_type": "n", "_value": [int(1000*random.betavariate(1,3)) for i in xrange(10000)] }
}
jsondata = json.dumps(data)
# Form the PUT request
requestHeaders = {"Accept": "application/json"}
req = urllib2.Request(httptrapurl, jsondata, headers = requestHeaders)
req.get_method = lambda: 'PUT'
opener = urllib2.urlopen(req)
putresponse = json.loads(opener.read())
# Print the data we get back to the screen so we can make sure it's working
print putresponse
print jsondata
print
# Wait a minute
time.sleep(60)
This will show up in Circonus as:
You can refer to the Circonus User Manual for more details about the HTTPtrap check. Also, please refer to the information there to import our certificate if you see the following error while following these instructions:
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>
