Grafana 2 and Nginx

I have spent a bit more time tuning that setup so it works the way I want(tm). I have also decided that I no longer want to use the default graphite web UI but rather use grafana exclusively. The collector used here is Diamond. It has an extensive list of collectors to grab all the metrics.

First, start by setting up your graphite site in the backend, I allow it to be reachable only locally because Grafana will proxy the queries to it. So create /etc/nginx/sites-available/graphite and fill it with:

server {

  listen 127.0.0.1:8082;

  server_name  localhost;
  root   /opt/graphite/webapp/graphite;
  index index.html index.php

  access_log  /var/log/nginx/graphite.access.log;
  error_log  /var/log/nginx/graphite.error.log;

  location / {
                gzip off;
   include uwsgi_params;
   uwsgi_pass   127.0.0.1:3031;
  }

}

You then need to install grafana where you see fit, in this example, I put it under /opt/grafana. Then create /etc/nginx/sites-available/grafana

server {

  listen 80;

  server_name  grafana.frlinux.net;
  root   /var/www/html;
  index index.html index.php

  access_log  /var/log/nginx/grafana.access.log;
  error_log  /var/log/nginx/grafana.error.log;

  location / {
   proxy_pass http://localhost:3000;
  }

You will of course still need to set up grafana, i will refer you to the official documentation.

Symlink the 2 sites under sites-enabled then restart nginx. Et voila !


Graphite and Nginx with uwsgi (2015 version)

A few years back, I wrote this tutorial to install Graphite with uwsgi on Debian. At the time, I used uwsgi 0.9.9 which has since evolved. My current Debian Jessie uses packages available from the system rather than having to use tarballs. It is now provided as version 2.0.7

Needless to say it has changed quite a bit. I have spent some time configuring it right and eventually got it. For posterity, here’s the config I now use, the rest of the configuration is still pretty much valid from the previous post.

Install the following packages: uwsgi uwsgi-core uwsgi-plugin-python
Create a file called graphite.ini in /etc/uwsgi/apps-available/ then copy the following into it, and symlink it to apps-enabled.

[uwsgi]
processes = 2
uid = www-data
gid = www-data
chdir = /opt/graphite/webapp
pythonpath = "['/opt/graphite/webapp'] + sys.path"
manage-script-name = true
mount = /graphite=/opt/graphite/conf/graphite.wsgi
socket = 127.0.0.1:3031

Restart the uwsgi service and check the logs, they should get created in /var/log/uwsgi/app/graphite.log by default.

Upgrading your nexus phone the adb way

Because I got fed up with OTAs and I also play too much with my phone, I decided to load up factory images directly with ADB. This process does not wipe the data for me, as long as you are careful on what you wipe.

In order to do this, you need to have your bootloader unlocked, if you unlock it now, you will lose all your data.

Here goes:

# check that you see your phone
fastboot devices
# get latest image
wget https://dl.google.com/dl/android/aosp/hammerhead-lmy47d-factory-6c1ad81e.tgz
# md5sum this shit
md5sum hammerhead-lmy47d-factory-6c1ad81e.tgz
# untar and get into it
tar xvf hammerhead-lmy47d-factory-6c1ad81e.tgz
cd hammerhead-lmy47d/
# unzip the different images
unzip image-hammerhead-lmy47d.zip
# flash all the shit
fastboot flash radio radio-hammerhead-m8974a-2.0.50.2.25.img
fastboot flash system system.img
fastboot flash boot boot.img
# reboot in recovery and flush cache and dalvik
# reboot and let it upgrade, profit.

6 months later with Tado

I posted a review of the Tado back in March and thought I would re-visit it after over 6 months of using it.

I would like to start with an important disclaimer. Never assume that the wiring is right if you moved into a place and are not quite confident with electricity. Back then I based my wiring on the existing one that was done for the timer. Turns out that it was incorrect. This is why when in summer time, Tado brought the water schedule functionality, it did not work for me. A friend came by to look at the wiring and figured out what was wrong. Thinking back, I got quite lucky that i managed to get the heating working. So long story short, a second set of eyes that actually understand electricity: good! Thanks Glen :)

Tado got out a new model with display now, which is not the model I have, just in case you stepped on this from a search and wonder what is coming next.

Tado completely rewrote the web interface and also the phone apps. I got to say, it is very welcome as the previous web interface was a bit clunky. The phone app was also in need of cleanup, just for speed alone. I am glad to report all additions actually helped.

Re-visiting what I think of the Tado, it has been a very good investment. Now that the wiring is fixed, I have thermostat as before but also water schedule when i want to, this is really cool for when the warm days will come back. I have advised a few friends to go for Tado and as far as I know, no one has been disappointed yet. Disclaimer: I am not paid by Tado to write this, I am just very enthusiastic about this device.

Importing sqlite3 to MySQL in a semi non painful way.

The example I will give below should work for most data. I needed to import dashboards for graphite in sqlite3 format to MySQL which is our now standard backend. This is the rough steps I used.

Get that file from /opt/graphite/storage/graphite.db then dump it like it is 1982:

sqlite3 graphite.db
sqlite> .output graphite.sql
sqlite> .dump dashboard_dashboard

Grab that wonderful python script. I am copy/pasting it here just in case in disappears.

#! /usr/bin/env python

import sys

def main():
    print "SET sql_mode='NO_BACKSLASH_ESCAPES';"
    lines = sys.stdin.read().splitlines()
    for line in lines:
        processLine(line)

def processLine(line):
    if (
        line.startswith("PRAGMA") or
        line.startswith("BEGIN TRANSACTION;") or
        line.startswith("COMMIT;") or
        line.startswith("DELETE FROM sqlite_sequence;") or
        line.startswith("INSERT INTO \"sqlite_sequence\"")
       ):
        return
    line = line.replace("AUTOINCREMENT", "AUTO_INCREMENT")
    line = line.replace("DEFAULT 't'", "DEFAULT '1'")
    line = line.replace("DEFAULT 'f'", "DEFAULT '0'")
    line = line.replace(",'t'", ",'1'")
    line = line.replace(",'f'", ",'0'")
    in_string = False
    newLine = ''
    for c in line:
        if not in_string:
            if c == "'":
                in_string = True
            elif c == '"':
                newLine = newLine + '`'
                continue
        elif c == "'":
            in_string = False
        newLine = newLine + c
    print newLine

if __name__ == "__main__":
    main()

Rename it something like sqlite2mysql.py then execute the following:

cat graphite.sql | python sqlite2mysql.py > graphite-mysql.sql

I would advise you add a few statements at the beginning and the end like:

USE graphite; 
...
COMMIT;

All you need to do now is to dump it back to MySQL like this:

mysql < graphite-mysql.sql

You should now have a full DB of dashboards in MySQL format.

Docker on Debian Wheezy

I recently got to play with Docker at work and thought, what would it take to get that running on a Debian stable system like Wheezy. In short, surprisingly little if you are not afraid to install a more recent, less tested kernel.

So I guess, let’s start and see what is involved. I decided to follow this Ubuntu installation guide directly from Docker. Due to a bug in Docker, kernel 3.8 or above is advised, so I installed the Debian backports repo. To do so, edit /etc/apt/sources.list and add:

# backports for new kernel
deb http://ftp.debian.org/debian/ wheezy-backports main non-free contrib
# Docker Repo
deb https://get.docker.io/ubuntu docker main 

Then install the newer kernel by doing:

apt-get update
# needed for the Docker repo
apt-get install apt-transport-https
# key for that repo
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
# install new kernel
apt-get -t wheezy-backports install linux-image-amd64 linux-headers-amd64
reboot # last step will reboot your box but you guessed it

Once you have rebooted, check your kernel version to make sure you rebooted on the right one. Then it is time to install the Docker repo and install the software. We edited the repo before and added the key for Docker, so all that remains to be done is:

apt-get install lxc-docker

At this point, you should see the Docker service starting by itself and have a working docker host ready to install all the wonders in the world. You can do a quick check by typing:

# docker version
Client version: 1.1.1
Client API version: 1.13
Go version (client): go1.2.1
Git commit (client): bd609d2
Server version: 1.1.1
Server API version: 1.13
Go version (server): go1.2.1
Git commit (server): bd609d2

If you need convincing on what Docker can do for you, head to their official documentation to get started. I would also suggest you take 10 minutes to look at Docker at Spotify

Enabling HDMI sound on an ion2 motherboard for XBMC

A friend of mine gave me a nice home made HTPC. I had forgotten the joy of setting up sound and video on Linux. I ended up installing Ubuntu 14.04 LTS with the latest XBMC 13.0 but the part that really was troublesome was to get the sound working.

Turns out PulseAudio tries to be too clever about this, it detected the wrong output. I eventually found a solution on this thread.

In short, edit /etc/pulse/default.pa and added this line:

load-module module-alsa-sink device=hw:1,7

Of course, this will depend on your device, to see what you have, do aplay -l

Installing Skyline to monitor graphite

I had heard of that project a while back and was curious to give it a go, with this weekend, I ended up having a bit of time to test this. Skyline is a project created by etsy which is designed to monitor automatically graphs and detect anomalies.

I would have normally used a CentOS 6 installation to test this but it turns out that the requirements are actually quite important in terms of dependencies. You will need fairly recent versions of packages to make this work. This is why I ended up using Ubuntu (14.04 LTS at the time of writing).

You can choose to use pip in order to install dependencies but I preferred to use the distribution’s packages instead. If that works for you, here’s what you need to install:

apt-get install python-numpy python-scipy python-pandas \
python-patsy python-statsmodels python-msgpack \
python-unittest2 python-mock python-simplejson \
python-hiredis redis-server python-daemon python-flask

Next, you need the latest version of skyline, you will need git installed for this, you can just do:

apt-get install git-core 
cd /opt
git clone https://github.com/etsy/skyline.git
cd skyline
cp src/settings.py.example src/settings.py
cp src/redis.conf /etc/redis/ # copying redis skyline config
mkdir /var/log/skyline /var/dump /var/run/skyline
chown -R redis /var/lib/redis/ 
service redis-server restart # important

You will need to modify the port and address if like me you are not using the same machine, then you can start the daemons. There is two things I fell into when i started them: first, make sure your host has its correct name and ip in /etc/hosts, horizon will get upset if you don’t. Second, also make sure that in the settings.py you will have to replace 127.0.0.1 by 0.0.0.0 if you want to be able to connect to the webapp outside of your machine. Last but not least, change the value of the http interface to point to your graphite instance, failing this, not much analysis can happen. Also note that you have to run the web interface on port 80, using anything different will fail.

Time to start it all:

cd /opt/skyline
bin/horizon.d start
bin/analyzer.d start
bin/webapp.d start

If at any point starting these daemons you have an issue, you are on your own, use the logs. I have included in the commands above all the issues I had, so you should be alright starting all the daemons.

You will need to direct your graphite metrics to a relay for duplication, this is nicely explained here.

You can also do a check to make sure it’s all good, luckily for you, the project includes a little utility to test this.

/opt/skyline# python utils/seed_data.py
Loading data over UDP via Horizon...
Connecting to Redis...
Congratulations! The data made it in. 
The Horizon pipeline seems to be working.

Installing Suricata 2.0 on Debian with JSON support

A few months ago, I wrote a blog entry worth reading if you lack context on installing Suricata on Debian with banyard and syslog support. During my original research, Suricata 2.0 was under development but has since been released. What is very interesting about this is its JSON support.

I am using Debian Wheezy but I did not want to install from sources nor upgrade to sid. So I ended up recompiling libhtp1 and suricata 2.0 for wheezy, which you can now download right here:

These packages have been compiled on my own wheezy server with latest patches installed. I have also installed libjansson4 and compiled suricata 2.0 with JSON support (Debian suricata on wheezy disables JSON support by default).

It will then allow you to output JSON directly to a file, I advise you to take a look at that link on Logstash Kibana and Suricata JSON output if you need more information on how to enable this with your logstash/kibana installation.

You can install some nice dashboards for suricata to use in Kibana, I have been using these ones lately.

I will write a bit later on tidying your alerts.

A week with the Tado

Well almost, in 2 days it will be, but I feel I have enough evidence to write up a quick review of the beast. So Tado is a concurrent to nest which you can purchase for most houses/apartments depending on the boiler you have. It is a thermostat unit for your boiler. It is highly recommended you go through the site to confirm that your setup is compatible.

My device took a while to arrive because after Christmas, Tado became really popular and they started to have an impressive (several weeks) backlog of orders. Eventually I received this.



I had a little trouble with the wiring due to the fact that my house is not exactly what you would call standard, so when I followed the wiring examples from the website, it just did not work for me. This is also because I can only heat water at the same time as the boiler is on, there is no separated pipes. After dealing with support, they were a bit slow in providing me with the right wiring, I ended up giving a call to my electrician to figure this out, the result being here.


Once this is wired right, you can put the cover back on and check on the website that all the bits are working. In fairness, apart from the minor setback in wiring, it is a piece of cake to install. You register the device with the user/password written on the card in the box.
You can then setup our phone or any other tablet to register with tado, this will allow tado to use geo location to save on power/heating when you are outside your home. The website interface reports at all times on the current temperature inside and also the weather report that it is supposed to use for intelligent heating.


The reporting is fairly straight forward, it is able to tell you at a glance how much heating it had to do (dark blue patterns on the graph), how long you were away on a given day and other useful information. For heating, there are two main settings for the device: savings or comfort. Savings is when you want to spend at least as possible on your bill, so it will not heat straight away when it detects you are moving home whereas comfort will be much snappier to react. I chose savings because I do not mind that the house will take slightly longer to heat up.


You can also set temperatures for normal operations and sleep, on the web interface or your phone. The phone application is quite responsive and intuitive, it will notify you of current status and will change colors to let you know which mode is it in, like below in away mode when you step out of your place.


It will also change context when the night comes, depending on when you have set up your sleeping hours. I have decided to go for custom times depending on the days as I do not wake up at the same time regularly. You can also override all settings to manual and just heat up the place if you need to, there is a button on the device itself, or you can control this using the application or the web interface.


I bought the tado to finally replace my timer with a clever thermostat and it does just that and very well. In the end, I am unsure if it will cost me a bit more than it used to mostly because i used to heat up the house 3h per day, and I think at the moment it heats up for slightly longer than that. That said, having a house that is at the right temperature all the time and reporting on energy consumption is quite good.

Very happy about the purchase. I have asked about a timer functionality for summer so i can heat up my water a couple hours a day but no response so far.