How to save your monitoring metrics in Prometheus vs Graphite.

Hey everyone in this article of tooling in DevOps we will talk about how we can save our monitoring metrics and use them to visualize in fancy graphs.

Let’s say we have metrics of how much time our servers are taking to serve any request. Now we want to save it in some time series database so that we can use it later to plot it. What mechanism we can use to save this data and which is useful in that scenario.

Pull vs Push

We have two mechanisms to save our data in our database. In the push mechanism, we try to push our data in the database whenever it is available. In pull mode we just expose our data to our database for consumption and our database may keep polling for this data and then saves it in the next attempt.

Where Pull is useful?

This is useful when you want your database to decide when to get the data.

This need a continuos server running to expose your metrics on a port which will keep running to which your database can get the metrics.

Where Push is useful?

Push is useful mostly in short running tasks where you don’t to run a continuous server to expose the metrics to the database.

What you can use for pull-based model?

Prometheus is very useful for pull-based model. You can expose your metrics on a server and port and define about the endpoint in your Prometheus server and it will pull the metrics on intervals you have defined.

What you can use for push-based model?

Graphite is a good option if you want to push the metrics from your application. Graphite exposes a port to which you can send the data and it will save it.

Python Code to expose data to Prometheus

from prometheus_client import start_http_server, Summary
import random
import time

# Create a metric to track time spent and requests made.
REQUEST_TIME = Summary('request_processing_seconds', 'Time spent processing request')

# Decorate function with metric.
@REQUEST_TIME.time()
def process_request(t):
    """A dummy function that takes some time."""
    time.sleep(t)

if __name__ == '__main__':
    # Start up the server to expose the metrics.
    start_http_server(8000)
    # Generate some requests.
    while True:
        process_request(random.random())

This is a simple example taken from Prometheus python client. You can run this and it will expose data to be consumed. You can see it in your browser using localhost:8000

Python Code to send data to graphite

import graphyte
graphyte.init('graphite.endpoint.com', prefix='system.sync')
graphyte.send('foo.bar', 42)

It seems far simple to push the code by this. By default, it pushes the data to 2003 port.

You can learn how to setup graphite here.

Now you can visualize this code using Grafana. We will talk about Grafana in the next article and try to draw some fancy graphs.

Till then subscribe and stay updated.

---