The Things Network (TTN) is a global LoRaWan public network kernel based on crowd-source infrastructure. This initiative sounds really interesting in my point of view because it breaks one of the main LoRaWan issues : the country based operators organization.

Related to this good point, the negative point is the current area covered by a such network limited to the contributor localization, generally in the main cities.

That said, I’m convinced : when you want to create a private LoRaWan network you have to manage a network kernel managing gateways, device authorizations… You can do it yourself, you can pay someone to do it for you or, eventually, you can lever TTN for this use. As a counterpart you will extend the network and offer this benefit to any around. That way it makes sense and gives large opportunities to the networks and it’s private users.

Long introduction for a technical topic … How to join this network once you have a Kerlink LoRa Iot station available ?

Disclamer

As usual, but particularly on this post : you follow all of this at your own risk

 

Configure the kerlink gateway

First of all my gateway already had a SPN v4.0.1 installed, the tutorial of TTN seems to be made for a factory fresh kerlink gateway with no SPN deployed on it. I found no way to change the SPN by TTN forwarder. And I was not able to restore the gateway to the factory defaults. In fact more than that by trying multiple operation to restore it I finally crashed all the gateway and I had to return it to Kerlink for full restore.

After getting some access to the documentation (one part I wasn’t noticed and had access previously I read you can’t un-install SPN v4.x actually).

So … If you are SPN 2.x or 3.x, take a look to the Kerlink wiki for uninstalling. If you are SPN 4.x I would tell, you can’t move it to TTN. My advice, be careful, uninstall is not well package and as you have no root access on gateway once SPN is install… it’s a mess.

That said, this post will describe a fresh install with no SPN installed on the gateway. So my situation is, be careful if different :

• PROD_FW=wirmaV2_wirgrid_v2.2
• KERNEL_VER=3.10.37-klk-d26c52

You can run the following command to get these information

# get_version -u

Install the TTN packet forwarder

One of the good point of TTN is the pretty nice documentation you can find and the tutorial is accessible following this link. Before following this link I recommend to take a look to the next part as it did not worked as expected for me.

Basically you have to copy the different files into an USB drive like this:

• dota_thethingsnetwork_v1.3_EU.tar.gz is obtained from the given link.
• produsb.sh is the content of the produsb.zip file obtained from the given link
• md5 is the result of md5sum shell command on dota_the… file you need to create this file
• usbkey.txt is the key obtained from kerlink to be able to deploy anything on your gateway

Once you have created this usb drive content, you can plug it on the gateway and wait for the update to be executed. The best is to have a debug cable connected to see the update process in execution.

At end of the process you can login on serial console or ssh using the root/root account.

you can check the installed components in the gateway directory:

/mnt/fsuser-1/thethingsnetwork

Get ready to use TTN

You can check the configuration :

• global_conf.json contains all the radio settings : frequency and spread_factor
• local_conf.json contains the information for the packet forwarder specific to TTN

In this last one you have

• the Gateway_ID equivalent to EID <= Note this ID you will need it after
• server ports
• server address

You also need to have your gateway accessible from the web on port 1700, for this you need to configure your LAN router to open and forward this pour  1700/UDP to the address of your Gateway. If you are using a 3G connectivity you need to have a bi-directional connectivity with a public IP address.

You can check it is communicating using tcpdump

# tcpdump -n -vvvX host 52.169.76.203

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
21:50:41.270391 IP (tos 0x0, ttl 64, id 3609, offset 0, flags [DF], proto UDP (17), length 40)
    10.0.0.167.59212 > 52.169.76.203.1700: [udp sum ok] UDP, length 12
    0x0000:  4500 0028 0e19 4000 4011 a091 0a00 00a7  E..(..@.@.......
    0x0010:  34a9 4ccb e74c 06a4 0014 beb8 0110 b902  4.L..L..........
    0x0020:  0000 024b 0805 029f                      ...K....
                             \s
21:50:41.607876 IP (tos 0x0, ttl 64, id 3610, offset 0, flags [DF], proto UDP (17), length 183)
    10.0.0.167.44991 > 52.169.76.203.1700: [udp sum ok] UDP, length 155
    0x0000:  4500 00b7 0e1a 4000 4011 a001 0a00 00a7  E.....@.@.......
    0x0010:  34a9 4ccb afbf 06a4 00a3 5d16 014c e800  4.L.......]..L..
    0x0020:  0000 024b 0805 029f 7b22 7374 6174 223a  ...K....{"stat":
    0x0030:  7b22 7469 6d65 223a 2232 3031 372d 3032  {"time":"2017-02
    0x0040:  2d32 3120 3230 3a35 303a 3431 2047 4d54  -21.20:50:41.GMT
    0x0050:  222c 226c 6174 6922 3a34 352e 3739 3131  ","lati":xx.7911
    0x0060:  352c 226c 6f6e 6722 3a33 2e30 3934 3831  5,"long":xx.09481
    0x0070:  2c22 616c 7469 223a 3337 352c 2272 786e  ,"alti":375,"rxn
    0x0080:  6222 3a30 2c22 7278 6f6b 223a 302c 2272  b":0,"rxok":0,"r
    0x0090:  7866 7722 3a30 2c22 6163 6b72 223a 3130  xfw":0,"ackr":10
    0x00a0:  302e 302c 2264 776e 6222 3a30 2c22 7478  0.0,"dwnb":0,"tx
    0x00b0:  6e62 223a 307d 7d                        nb":0}}

Configure the Cellular access

This tutorial explain how to configure the 3G connectivity : https://www.thethingsnetwork.org/docs/gateways/kerlink/cellular.html

Basically you need to edit the /etc/sysconfig/network file to setup the Cellular provider settings.

# Selector operator APN
GPRSAPN=m2minternet        <= depends on your Internet provider
# Enter pin code if activated
GPRSPIN=
# Update /etc/resolv.conf to get dns facilities
GPSDNS=yes
# PAP authentication
GPRSUSER=        <= depends on Internet provider
GPRSPASSWORD=

# Bearers priority order
BEARERS_PRIORITY="ppp0,eth0,eth1"

If your GPRS connection does not requires Login / Password you need to modify the /etc/init.d/gprs by modify this line :

 [ ${GPRSUSER} ] && PPP_OPTIONS="user ${GPRSUSER} password ${GPRSPASSWORD}"

Then you need to modify the file /knet/knetd.xmlby adding some lines:

<!-- ############## connection parameters ############## --> 
 
<!-- nb of second to retry to connect to server if connection failed--> 
<CONNECT retry_timeout="10" /> 
<!-- port nunmber for local application kms connection --> 
<CONNECT kms_port="35035" /> 
<CONNECT auto_connection="YES" /> 
<!-- frequency of connection monitoring -ping- (in seconds) --> 
<CONNECT link_timeout="30"/> 
<!-- DNS servers will be pinged if commented or deleted. Some operators can block the ping on there DNS servers -->
<CONNECT ip_link="194.151.228.34"/> 

<!-- ############## default area for connection policy ############## --> 
 
<AREA id="default"> 
<ACCESS_POINT bearer="gprs" /> 
</AREA>

Transmission over ppp can be verified with the following command:

# tcpdump -i ppp0 -v

Register the gateway on TTN

The gateway is registered on TTN going on your profile then clicking on console – then you can select gateways and click on “register a gateway”

You need to select “packet forwarder” option, then enter the GatewayID obtained from the previous step, a description (whatever you want), select the European (depends where you are) plan and click on the map to select you gateway location. At end you indicate if you are indoor or outdoor. That’s it !

This done the gateway is up & running on the network

You should see you gateway as active in the gateway page after this.

For curious people you can also list the gateway and find yours on this page and see how many gateway are active on the network ( in feb 2017 it was about 1000 ).

Create a device

I have the The Things Uno device for making a first test but as it is basically a RN2483 + an arduino you can easily make your own device based on a such hardware. This page is the Quick Start Guide for The Thing Uno.

This first program will allow you to get your device EUI

#include <TheThingsNetwork.h>
#include <Wire.h>
#define loraSerial Serial1
#define debugSerial Serial
#define freqPlan TTN_FP_EU868
TheThingsNetwork ttn(loraSerial, debugSerial, freqPlan);

void setup() {
  loraSerial.begin(57600);
  debugSerial.begin(9600);
    
  // Wait a maximum of 10s for Serial Monitor
  while (!debugSerial && millis() < 10000);
    
  debugSerial.println("-- STATUS");
  ttn.showStatus();
}

void loop() {

}

In the Arduino console you will see :

-- STATUS
EUI: 0004A30B001XXXXX
Battery: 3273
AppEUI: 0000000000000000
DevEUI: 0004A30B001XXXXX  <== device EUI
Data Rate: 5
RX Delay 1: 1000
RX Delay 2: 2000

Now you can follow the device registration process as described in the quick start guide. From this procedure you will obtain the APPEUI and the APPKEY, now the device you be ready to register the network.

So the code may looks like :

#include <TheThingsNetwork.h>
#include <Wire.h>
#define loraSerial Serial1
#define debugSerial Serial

const char *appEui = "70B3D57EFXXXXXX";
const char *appKey = "6484D4207099A8FB2XXXXXXXXXX";

#define freqPlan TTN_FP_EU868
TheThingsNetwork ttn(loraSerial, debugSerial, freqPlan);

void setup() {
  // put your setup code here, to run once:
  loraSerial.begin(57600);
  debugSerial.begin(9600);
    
  // Wait a maximum of 10s for Serial Monitor
  while (!debugSerial && millis() < 10000); 

  debugSerial.println("-- STATUS");
  ttn.showStatus();

  debugSerial.println("-- JOIN");
  ttn.join(appEui, appKey,2,1000);

}

void loop() {
  debugSerial.println("-- LOOP");
    
  // Prepare array of 1 byte to indicate LED status
  static byte data[10] = { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A };

  // Send it off
  ttn.sendBytes(data, sizeof(data));
      
  delay(30000);
}

This will send a message containing 1 byte with value 2

And ! it works ! you can see it real time in the TTN console.

You can also see the metadata by clicking on each of the data:

{
  "time": "2017-02-22T19:35:57.59794586Z",
  "frequency": 868.1,
  "modulation": "LORA",
  "data_rate": "SF7BW125",
  "coding_rate": "4/5",
  "gateways": [
    {
      "gtw_id": "eui-0000024b0805029f",
      "timestamp": 2474258579,
      "time": "2017-02-22T19:35:57.567278Z",
      "channel": 0,
      "rssi": -83,
      "snr": 8.5,
      "rf_chain": 1,
      "latitude": 45.xxxx,
      "longitude": 3.xxxxx,
      "altitude": 367
    }
  ]
}

MetaData are really usefull and compared to the public network you have many more information like the Localization of the antenna. Usually this information is a secret. It can be useful to help you to localize your device.

Manage Backend callback

There are one think I REALLY LOVE in TTN : for your application you can create a javascript function to manage your payload decoding !!! so nice !

Thank to this you can create a nice JSON for you backend application !!

You go to Application >> Payload function

You have 3 step you can configure

• Decoder => transform your byte field message into a javascript object
• Converter => convert the previously decoded data
• Validator => check converted data, accept or refuse them.

This method allow you to manage a complex protocol with many different frame type easily.

In the decoder we can put something like this, and test the result in the UI :

Once done, you can see your data decoded directly in the backend dataflow.

Now, you can create an integrator :

You can store the data in a database, create an HTTP callback of call an IFTTT service ! I’ll choose an HTTP Integration I’m more familiar with : This choice will just ask you for an URL for the callback and some information for downlink if you want a bidirectional communication.

With a simple php file like this on your server side you can receive the data

<?php
       $_fileName = "data/ttn.json";
       $_file = fopen($_fileName,'a');
       $postdata = file_get_contents("php://input");
       fwrite($_file, $postdata);
       fwrite($_file,"\n");
       fclose($_file);
?>
<html><body></body></html>

The data looks like this

{ "app_id":"disk91test",                // given app name
  "dev_id":"disk91-ttn-uno",            // given device name
  "hardware_serial":"0004A30B001BXXXX", // device EUI

  "port":1,                             // comm port
  "counter":55,                         // frame sequence number
  "payload_raw":"AQIDBAUGBwgJCg==",     // base64 payload
 
  "payload_fields":                     // remember this is the decoder result
     {"bit":[0,1,0,0,0,0,0,0],
      "value1":1,
      "value2":3
     },
   "metadata":                          // MetaData as seen previously
     {
       "time":"2017-02-22T20:02:12.416080072Z",
       "frequency":868.5,
       "modulation":"LORA",
       "data_rate":"SF7BW125",
       "coding_rate":"4/5",
       "gateways":[
           {"gtw_id":"eui-0000024b0805029f",
            "timestamp":4049075219,
            "time":"2017-02-22T20:02:12.383737Z",
            "channel":2,
            "rssi":-59,
            "snr":7.8,
            "rf_chain":1,
            "latitude":45.XXXX,
            "longitude":3.XXXX,
            "altitude":365
           }]
     },
    "downlink_url":"https://....."}

The data contains all what we need, well presented, easy to use ! What I was not able to test is what is happening when the message is captured by different gateways : do we have multiple messages or does TTN wait for a certain time to create the callback waiting for all the gateway to return the message.

Conclusion

I’m was really happy making this test and see all the work the team have done, it is an excellent work, backend is the best I’ve seen and far away from the public one I’ve tested.

My point of view is TTN is a really good solution if you want to make your own private LoRaWan network because you can benefit of other antenna for free and access a high quality network kernel to help you for managing your device and antenna.

Side note:

• To restore the Kerlink gateway to the factory default, you had to execute the factory default procedure : push reset button 22 times with a 5-10 seconds wait between each. Basically if you have the serial console connected you click on reset button as soon as you see kernel booting. After 22 times the boot will start the restore procedure. This procedure takes then about 5 minutes. But in my case the SPN software were still here because it reinstall all the existing package on the gateway (even the borken one)