Lab 4: MQTT

Before you start !!

Register for a flespi.io account https://flespi.io/#/ You should not give any financial details (e.g. a credit card).

1. Overview

As discussed in lecture, MQTT is a widely used protocol for IoT applications; it is based upon a publish-subscribe model in which “things” and “applications” communicate by publishing and subscribing to topics through an MQTT broker.

In this lab, you will be communicating with a variety of MQTT brokers as you “ramp-up” your understanding. For example, initially, you will use command-line tools to publish and subscribe to a web-based test broker. Because this is a completely open server, and hence insecure, it is not an acceptable long-term solution. Thus you will then experiment with running a local MQTT broker – mosquitto – and finally a free commercial broker – flespi.com. In the latter case, you will use a host generated token to secure your communication.

In addition to learning how to communicate with MQTT brokers, you will also learn how to build a “virtual” IoT device using node-red. Finally, as a homework assignment, you will create a “virtual” IoT device using the language of your choice (Java, Python, etc) and the Paho MQTT libraries (https://www.eclipse.org/paho/)

Deliverables

You will need to create a lab4 directory as you did for lab3 by copying the node-red-template directory into lab4

You should perform all your work in your repo in the lab4 directory. This work includes node-red flows and the code developed for your homework assignment (see the end of this document). Your primary deliverable (in addition to the code/flows in your repo) is a report that should contain:

A description of the work you performed with any issues that arose.
For each section of the lab.
- Traces/screenshots as appropriate to document your work.
- Explanations of any flows or code that your wrote.

I will not be grading the code/flows you develop directly, rather, you will be graded on your ability to describe their function. It is not necessary to be verbose or to describe every line; your objective should be to describe, for a code-literate reader, what the various components do and any notable design decisions. Include a discussion of your homework problem in your report.md file.

The grading rubric for this lab is

Completion of tasks: 60%
- Communication with MQTT server using command line : 10%
- Communication with MQTT server using node-red: 10%
- Building a “thing”: 10%
- Communication with Flespi: 10%
- Securing your flow: 10%
- Building a Paho Client: 10%
Lab report: 40%
- Overview: 5%
- Steps taken + issues: 15%
- Discussion of your designs 15%
- Screenshots of flows/dashboards: 5%

2. Command line tools

The lab computers should have two command-line tools for MQTT sub/pub installed.

mosquitto_pub : for publishing to MQTT servers
mosquitto_sub : for subscribing to MQTT servers

You should test each of these by executing them with the “–help” option.

$ mosquitto_pub --help |less
mosquitto_pub is a simple mqtt client that will publish a message on a single top
ic and exit.
mosquitto_pub version 1.6.7 running on libmosquitto 1.6.7.

Usage: mosquitto_pub {[-h host] [-p port] [-u username] [-P password] -t topic | 
-L URL}
                     {-f file | -l | -n | -m message}
...

$ mosquitto_sub --help |less
mosquitto_sub is a simple mqtt client that will subscribe to a set of topics and print all messages it receives.
mosquitto_sub version 1.6.7 running on libmosquitto 1.6.7.

Usage: mosquitto_sub {[-h host] [-p port] [-u username] [-P password] -t topic | -L URL [-t topic]}
...

Notice that each of these takes a set of parameters, the most notable of which are:

-h : mqtt host to connnect to. Defaults to localhost
-t : mqtt topic to publish/subscribe to
-m : mqtt message body for publication
-v : print published messages verbosely

We’ll begin by subscribing to a topic on a public test server – test.mosquitto.org. Every MQTT server publishes basic system statistics/information that you can access with a subscription. For example, to access message statistics execute the following in a terminal window:

$ mosquitto_sub -h test.mosquitto.org -t '$SYS/broker/messages/#' -v

You should see a log of messages like the following:

$SYS/broker/messages/stored 88070
$SYS/broker/messages/received 754123259
$SYS/broker/messages/sent 5307375028
$SYS/broker/messages/stored 88548
$SYS/broker/messages/received 754129965
$SYS/broker/messages/sent 5307406929

You can cause mosquitto_sub to exit by typing ctl-c in the terminal window where it is running.

Notice that the statistics are periodically updated. The following https://mosquitto.org/man/mosquitto-8.html describes both the system messages and the syntax for subscriptions. Some of this information is summarized here https://github.com/mqtt/mqtt.github.io/wiki/SYS-Topics.

In the next step, you will subscribe to a topic for messages that you (and your classmates) will generate separately. Open a second terminal window. In one window, subscribe to the topic `po181u/#‘. Now generate messages – for example:

mosquitto_pub -h test.mosquitto.org -t po181u/subtopic -m 'a message'

where “subtopic” is some string that you pick (don’t use personally identifiable information). Practice sending messages with different contents, for example:

3.14159
danger will robinson
{"temperature": 24.0, "humidity" : 53.0}

3. Communicating with an MQTT server using node-red

Begin by copying the “template” into your lab4 directory. From within the lab4 directory execute

$ cp -r ../node-red-template/* .

Now start node-red from a terminal within the lab4 directory.

$ ./start-node-red

Create a flow called “MQTT Test1” with two nodes:

Now configure the MQTT input node. (QoS = 0, topic = po181u/#, …)

Configure the debug node to display the complete message. After deploying the flow, test it with the mosquitto_pub tool.

Finally add a pair of nodes to your flow and test them:

You should set the topic in the inject node and leave the topic in the MQTT out node blank. Make sure that you can send and receive MQTT messages in your flow.

4. Local Mosquitto Server

Now you will start a local mosquitto server. First create a file in your lab4 called “mosquitto.conf” with the following contents

bind_address localhost

Now start a mosquitto server in terminal from within your lab4 directory.

$ mosquitto -c mosquitto.conf

You can test this as:

$ mosquitto_sub -h localhost -t '$SYS/broker/messages/#' -v

Once you are convinced the server is working, copy your nodes to a new flow called “MQTT Test2” and change the broker in the MQTT nodes to localhost – you’ll need to add a new broker. (hint: copy/paste ⌘C/⌘V works !). Now test your flow.

5. Building a “thing” with node-red

It’s time to build a simulated “thing”. Copy your “MQTT Test2” flow to a new flow called “MQTT Thing”.

Reconfigure the inject node so that it does not set the topic, but does “fire” every second. Insert a function node between the inject and mqtt output nodes. Following the directions from last lab, write a function that sets the message to a random number in the range 18.0…24.9 with one decimal point accuracy. Also set the topic to “po181u/temperature” in the function and name the function temperature. The resulting flow should look like:

6. Web based broker (flespi)

When you log in to flespi, you should see a “home” screen that looks something like:

Select the MQTT board and add a publisher and a subscriber:

Change the publisher topic to po181u/test.
Change the subscriber topic to po181u/#.
Now subscribe ( ▷ symbol in upper right of subscriber window) and send a few test messages from the publisher (similar symbol in upper right).
Return to the main window ( < symbol in the upper left) and select the “Tokens” window.
Create a token (+ in the lower left) Set the Expire field some time in the future.

The token, which you can copy with the clipboard symbol, is used as a “user name” – no password is necessary.

You can test your connection with mosquitto_sub

$ mosquitto_sub -h mqtt.flespi.io -u <your_token> -t 'po181u/#'

and with mosquitto_pub

Copy your “thing” flow to a new flow called “MQTT Flespi Thing” and change the broker to point to flespi * host: mqtt.flespi.io * port: 1883 * username (under security): your token * password (blank)

Test your connection.

This isn’t completely secure because it’s using plaintext (1883) vs encrypted (8883) so a “listener” (man-in-the-middle) could learn your token.

7. Making your flow (more) secure

As mentioned above, using plaintext for the connection isn’t secure – both the token and the data could be exposed to listeners. The solution is to do everything over an encrypted (TLS) channel. Here we’ll implement the minimum level of security – using a standard TLS certificate to enable the use of a secure channel. Later in the semester we’ll discuss security more deeply.

Copy your “MQTT Flespi Thing” flow to a new flow “MQTT Flespi Thing TLS”.

Create a new flespi host (as above) named flespi-tls.

host: mqtt.flespi.io
port: 8883
Enable secure (SSL/TLS) connection
username (under security): your token
password (blank)

You’ll need to add a TLS configuration

Set the CA Certificate to ca-certificates.crt which should have been included when you copied the template to your folder

Test the flow.

8. Homework – Building MQTT clients with Paho (homework)

Your homework is to build a pair of clients – a publisher and a subscriber – in the language of your choice using the paho libraries – for example

python : https://www.eclipse.org/paho/clients/python/
java : https://www.eclipse.org/paho/clients/java/
go : https://www.eclipse.org/paho/clients/golang/
…

Be careful with the example code given – I found the python example had the wrong number of arguments for “on_connect”. (it should have been def on_connect(client, obj, flags, rc))

Your publish client should simulate a thermometer (as your node-red code did) and your subscriber should subscribe to “po181u/#`. In both cases your client should connect to your flespi account.

You do not need to use the encrypted (8883) port for this homework. It’s fine to use the (1883) unencrypted port. Getting TLS configured is an extra step that may not be worth the effort just to see how the Paho client software works.

It is generally considered insecure to publish tokens in source code. One solution is to accept the token as a command-line argument to your program. Alternatively you can just create a token that expires “soon” and not worry about this yet.