Announced during TheThingsConference 2021 this January, TheThingsNetwork console is upgrading to v3. In fact it is really more than a simple update because this consist in merging TheThingsNetwork and TheThingsIndustrie in a common environment. This upgrade is really different than a traditional upgrade because it has an impact on any gateways and any devices ; a big impact anyone needs to manage. Here are some of the currently known impacts on the networks and the different components:

• Traffic from v2 is routed to v3 but v3 is not routed to v2
• Gateway are not migrated automatically
• LoRa Application/Integration needs to be re-created in v3 to communicate over v3
• Device LoRaWan session needs to be refreshed
• Coverage map are lost
• Gateways using gRPC protocol (TTN Packet forwarder won’t relay V3 traffic over V2)
• … some more to be discovered

As the Network is managed by many different people, and not really planned and organized by TheThingsIndustry, we are going to have disconnections and fields operations to make all of this back to life.

It concerns the TTN Initiators (people deploying the network) but also the TTN users because they need to update Applications and restart/rejoin devices once the V2 has disappeared.

The currently know planning for Europe is (Other countries not yet planned)

• April 2021 – v2 console will be read-only (you won’t be able to add devices / gateways)
• September 2021 – v2 infrastructure will be switched off, no more communication on it

This blog post details my advises and experience of the migration. As I’ll do it in different waves for practical reason, this post will be updated. So please come back and refresh all along the months of February-March 2021 to get the updates, content and advice will be updated according to my progress.

Personal current advice about migration: START INVESTIGATING SINGLE DEVICE MIGRATION

I had some difficulties to migrate my first device and this has been solved by upgrading the gateway and redefining it in the TTNv2 backend. Discovering such situation earlier is important to get on time for upgrading. I recommend to start migrating 1 device to identify such situation. Migrating more than a device could still means impossible ways for some of your devices to communicate through v2 infrastructure. But it will help to prepare your infrastructure for migration.

Get support

TheThingsNetwork forum is the right place to find answers to many of your questions and ask for the one not yet answered. There is a special section here.

Once you are in the console, you can find the documentation link on the bottom right (thank you decartes for indicating it to me, I’ve spent the two first hours looking on documentation on Google and missing that link ! You can also use that link to access the documentation It was not too much complicated to find but you need to look at TheThingsIndustries instead of TheThingsNetwork… So, now you got the point, it is not an upgrade, it is a migration from TTN to TTI.

Device owner, prepare the transition

During the migration phase, the devices need to join the v3 infrastructure by opening a new session. As a consequence your device need to have a new join. We will see later how to migrate them. At first you need to ensure your devices will be able to rejoin automatically. A correctly developed device should request for a new join after a certain time not receiving any ack.

The problem is, nothing in the LoRaWAN spec details what is the best way for doing this (for real, there is no best way for doing this it really depends on your application). So you need to see what it is going to happen. Some devices will need to be manually restarted to create a new session. For some other, the disconnection discovery will be multiple days. It is better to know it and anticipate the impacts.

If you have written the device firmware, it’s time to implement a disconnection discovery with an automatic reconnect to support the network migration. Honesty, I really don’t like writing something like “you should write something in your firmware for a specific and should never happen event…” by the way, the idea is to support disconnection with an acceptable reconnect time taking into consideration you don’t want later to reconnected on every gateway up & down because you have a higher risk to not be able to reconnect and loose more messages.

When implementing reconnect, you should read the back-off process documented in the LoRaWAN specification: implement a random timer between join, apply 1% duty cycle on join during the first hour, then 0.1% during the first 12 hours and 0.01% duty cycle after.

Update your Gateway firmware and stay in v2

Ensure your gateway will be able to communicate using basic station protocol. If not you must upgrade your firmware to support it. If your gateway is connected using the gRPC protocol (like some Laird) This gateway will not be able to relay the traffic to V3, even when declared in v2. You can do gateway upgrade right now.

This step is not about migrating the Gateways to v3. This is the last step to do and later is better.

Before starting, you need to know that there is a lot of communication protocol between gateways and TTNv2:

• Semtech legacy UDP protocol – unsecured but widely deployed
• gRPC – created by TTN to fix Semtech UDP protocol but discontinued
• mp-forwarder – evolution of Semtech legacy UDP
• Basic Station – Semtech new standard

This step is about going out of the gRPC, mp-forwarder and Semtech lagecy UDP protocols as much as possible. Ensure your gateway will be compatible with TTNv3.

The gRPC protocol is not suitable for TTNv3 (it was the default one in the old firmware versions of Laird gateways. I’ve faced issues with some gateways during the migration: even if your gateway is connected to TTNv2, the traffic from a gRPC packet forwarder based protocol are not be routed to v3. So such gateway could not relay communications from v3 devices.

The TTN Packet forwarder (mp-forwarder) is suitable for TTNv3, and should be routed from V2 to V3, but is not recommended.

You can take a look in the chapter above about gateway upgrade for the one I’m using and I had to upgrade.

If the gateway firmware supports basic station and you are in basic station you basically nothing have to do right now. If the gateway supports basic station and you are currently running the Semtech Legacy UDP packet forwarder, you can stay like this but I recommend to switch to basic station protocol staying in TTN v2. You can take a look to TTNv2 documentation pages. When the gateway supports Basic station but your packet forwarder is gRPC, you shall switch to basic station or at least to Semtech UDP legacy packet forwarder.

If your gateway firmware does not supports Basic station, you may upgrade to a firmware supporting it.

If your gateway have no option to upgraded to a firmware supporting Basic Station you will be able to connect using the Semtech UDP legacy protocol.

When switching from gRPC protocol to Semtech protocol, your gateway id will change and, as a consequence, the mapping data will get lost.

Migrate application

Application migration can be made without impacting anything. When reaching the integration, you may have to upgrade your application back-end endpoint to support the new format.

This is the first operation to be executed, migration of your application to v3 will allow to start the migration of your devices. If you need to manually restart a device to make it joining the v3 environment, you will be able to migrate it one by one.

The idea is to create the same application in the v3 environment, the v2 application can be kept if you make a device per device migration. During the join procedure, the system will search for a device in v2. If there the v2 session will be created (it is not our objective). If not there, the join request will be executed by v3 and the a v3 session will be created.

So once you have created the application in v3 with all your devices you can destroy the v2 application, or you can, device by device destroy them in v2 at the moment you do the manual reset. So that way you can avoid to loose messages.

Create a new application

Here are the steps to create a new application to receive the devices previously connected to v2:

This step is basically the easiest one. You just declare an application. In v2 the application ID was generated by application. This was basically the purpose of the ApplicationID. But with time it appears that this notion makes none sense (it has never been in fact, but it was the time needed by the community to understand that this notion makes non sense at the device level and just make sense at the backend level. Heritage of this time where LoRaAlliance learned IoT basics, we still have this ID and it is basically burned in the firmware of devices so we still have them and it’s like a 128bit device id to identify each devices. So… text to be around the picture terminated, the creation of the application is just a declaration in backend to associate common information around a group of device (oh… we are back on the Sigfox DeviceType concept…). Nothing more to do here. But we need to create the associated elements.

Duplicate the payload formatters

Here it start to be fun, the concept are the same. You have more options, so this is basically a good news. But, the name of variables are different. By the past you returned a structure directly your decoded content, now the structure is:

{
   data : { previous decoded payload },
   warning: [],
   errors: []
}

As a consequence, we can do the migration that way:

function Decoder(bytes, port) {
  var batC = 256*bytes[6]+bytes[7];
  if ( batC >= 32768 ) {
    batC = batC - 65536;
  }
  ...
  var decoded = {
    boardTemperature:boardTemp/10.0,
    batteryCurrent:batC/100.0,
    batteryVoltage:batVoltage,
    boardVoltage:(256*bytes[14]+bytes[15])/100.0,
    externalPowerState:bytes[18],
    batteryPower:batPower/100.0
  };
  return decoded;
}

This v2 integration can be rewritten in v3 like the following:

function decodeUplink(input) {
  var bytes = input.bytes;
  var port = input.fport;

  var batC = 256*bytes[6]+bytes[7];
  if ( batC >= 32768 ) {
    batC = batC - 65536;
  }
  ...
  var decoded = {
    boardTemperature:boardTemp/10.0,
    batteryCurrent:batC/100.0,
    batteryVoltage:batVoltage,
    boardVoltage:(256*bytes[14]+bytes[15])/100.0,
    externalPowerState:bytes[18],
    batteryPower:batPower/100.0
  };

  return {
    data: decoded,
    warnings: [],
    errors: []
  };

This is basically not too much difficult. The blue part is the same as in v2 with no modifications int it. If you want to get benefit from the new features, you can read the complete documentation available in TheThingsIndustries documentation.

If you add something in errors field, the message will be dropped.

Create the new integration actions

You need to release an update of your back-end application (when you had one) because the payload format has changed.

The payload format is a bit different between v2 and v3, your back-end application endpoint receiving TTN traffic must support these modifications.

You can take a look to the TTNv2 payload format and the TTNv3 payload format documentations.

On top of this, during the transition, the rx_metadata, for messages transmitted through the packet broker ( basically the messages coming from v2) will only contains one gateway named “packetbroker” and an extra “packet_broker” structure:

“rx_metadata”:[
 {
     “gateway_ids”:{
        “gateway_id”:“packetbroker”
     },
     “packet_broker”:{
        “message_id”:“01EYJTZ0Y5W30V5TV06TKXSC96”,
        “forwarder_net_id”:“000013”,
        “forwarder_tenant_id”:“ttn”,
        “forwarder_cluster_id”:“ttn-v2-eu-3”,
        “home_network_net_id”:“000013”,
        “home_network_tenant_id”:“ttn”,
        “home_network_cluster_id”:“ttn-eu1”,
        “hops”:[
             {
                 “received_at”:“2021-02-15T12:34:39.173092993Z”,
                 “sender_address”:“40.113.68.198",
                 “receiver_name”:“router-dataplane-696b87d68f-2nrhh”,
                 “receiver_agent”:“pbdataplane/1.2.0 go/1.1 ..."
             },
             {
               ...
             },
             {
               ...
             }
            ]
     },
     “rssi”:-63,
     “channel_rssi”:-63,
     “snr”:12.2,
     “uplink_token”:“eyJnIjoi ... ImZjaWQiOiJ0dG4tdjItZXUtMyJ9fQ==”
 }
],

Even if multiple v2 gateways receives the message you will have a single packetbroker entry. When mixing gateway V2 and V3, you will have the list of V3 gateways in the rx_metadata array and one entry for packet_broker. The gateway location is not provided. Device location neither.

The TTNv3 console allows to make integration test with the ability to simulate an Uplink. That way you can check the integration, even without devices communicating. Unfortunately, the message format is really simplified and you won’t be able to verify many things with this feature currently. As an example, the rx_metadata structure is not pushed over the uplink simulation. As a consequence you won’t be able to verify this critical part and you can also get some crash related to the missing part of the structure. So be careful when testing with this feature: I’ve spent 1 hour trying to fix crash when in fact the system was correctly working with normal device payload but not with simulated payload. At end my code is stronger but my time has been lost.

Migrate devices

I’ve been able to migrate one device from v2 to v3 successfully after having updated my gateway firmware and re-created the gateway in ttnV2 backend. The reason was the following: the gateway was using TTN packet forwarder based on gRPC protocol. This protocol is discontinued and not forwarded to V3.

Preparation step

I’ve been some issues when migrating devices connected to an old declared gateways (laird) with a old firmware (v93.7). This issue has been solved after upgrading firmware and redefining the gateway in TTNv2 backend. The redefinition was a mandatory step due to firmware upgrade (v93.8.5.21) and the gateway naming changed.

I recommend to migrate one single device and verify the entire connectivity before planning the whole fleet migration. You could declare all the devices into v3 environment. The communication from v2 stays good until they are disabled from v2. But if you do this, if some V3 gateways respond to device join, this device will become out of v2 architecture for this session. So one device is good enough right now.

Migrated device will loose the gateway meta-data when passing through the packet broker. If you were using the network information in your application like for positioning of mapping network it won’t work anymore until you switched the gateways to ttnv3.

Migrate devices

For each of the devices, you need to create a new one in TTN v3. Currently there is no batch tool for the migration. I assume you could create a tooling based on the API to export and import the devices.

The new version of TTN is really more powerful than the previous one with many settings and most of these information was available in the previous version, this is why you need to make a custom import. As an example for each of the devices you can select the device LoRaWAN version and that way get benefit of the sub version of the protocol the network server was not able to identified from communications.

So at first you can create the new device:

Then you have the choice between known devices or custom devices. As the device I’m importing are home made, I’m selection custom and in the next steps I’ll have to detail the device configuration. If your device is from a commercial brand, you can access templates to simplify the configuration. If you select “Other”, you need to click on the manual registration link.

Then you select the device protocol specifications; most of the devices are 1.0.2. The second screen I’ve printed in one later step after that one, but in my point of view more related and better fro post readability to add it here. In that second one you select the radio zone (I’m always disappointed to see that a device moving around the world need to be registered multiple times with multiple ids to work… hum .. basically it’s not working at the end … who talk about roaming blablabla in 2015 ?). Then select the regional parameter version ; this is basically concerning US900, INDIA865, KR290, AU915 and some details in class B.

Between these two steps you have the join configuration to setup. What you need to do is to copy and paste the values you have in the console v2.

After this, you will have you device registered. Until you deactivate the device on the console V2, this definition in V3 will be kept inactive.

It would have been cool to have a button in V2 to deactivate a device instead of having to delete it. But unfortunately, they did not propose that option. I do not recommend to destroy your device in v2 because if you made a mistake when doing cut & paste you would have lost your credentials and you will have no other way than getting the device from the field to reconfigure it.

As a consequence, my proposal is to modify the device in V2 to make it different and allow it to switch to v3 on the next join request. For this, you go to device settings in the TTN-V2 console and change the Device EUI (you can change the first byte by adding 1, the rollback will be easy and it’s enough to deactivate that device from v2).

This change does not disconnect the device until the next Join, so now, you can go in the field to make the restart (or wait to a planned rejoin) to switch. If your device can support a disconnection and will automatically reconnect, I don’t see another way to do it. (I’ve made a evolution proposal in v2 to simplify this…. let’s see if taken into account).

Once the device will have rejoin the network, you will see it on TTN-V3 back-end like this.

In the lave data frame, you can watch the last communications related to the device and see it joining.

Migrate Gateways

Do not yet migrate your gateways from v2 to v3 – this will prevent any devices on your coverage area to reach the network until they will be migrated. This will be the last step to do. Eventually if you are deploying some new gateway they could be deployed in v3.

Upgrade GATEWAYS to use BASIC STATION

The best choice to connect your gateways to TTNv3 (even v2) is to use Basic Station protocol. It can works two different ways:

• CUPS based – Configuration and UPdate Server. In this mode, the CUPS server will configure the LoRaWan Network Configuration layer. It will also give you access to some basic settings of the gateway remotely. This mode is not working on TTNv2 and is Beta in TTNv3.
• LNS based. LoRaWan Network Server. As a consequence this is the recommended mode for this migration.

To connect to LoRaWan Network Server (LNS) basic station you need to indicate the LNS server and associated certificate. For this reason you will not be able to switch from TTNv2 to TTNv3 without accessing the gateways one by one.

Certificate Type	TTN v2	TTNv3
Server Certificates	Page where to download it I only succeeded to connect with the trustid-x3-root.pem (validity 2021-03-16)	Page where to download it https://letsencrypt.org/certs/isrgrootx1.pem (validity 2035-06-04)
Client Certificate File	No needed	Not needed
Key file	Not needed	Need to be generated (will be detailed later)
LNS Server address	wss://lns.eu.thethings.network:443 change eu by us… for other zones	wss://eu1.cloud.thethings.network:8887 change eu by us… for other zones
CUPS Server adress	Not available	https://eu1.cloud.thethings.network:443
Semtech leagcy LNS endpoint	router.eu.thethings.network:1700 change eu by us.. for other zones	eu1.cloud.thethings.network:1700 change eu by us.. for other zones

If you want to make a one-shot switch, you can prefer the Semtech Legacy protocol. That way, you can use a personal domain name like gateways.disk91.com and use a CNAME to the TTN server. That way you can switch it remotely all in once, just changing the CNAME target.

router.eu.thethings.network

Update Laird Gateway’s Firmware and switch to basic Station

Making a such migration is a good opportunity to update the gateway’s firmware and switch of the old and unsecured old Semtech’s UDP protocol to basic station protocol.

For a Laird gateway, it is basically automated but the implementation is really bad… I first you may know you will lost all your settings. Then, the default link will correspond to a really old version (sometime the same as the one you already have. You need to change the default upgrade URL for:

https://connectivity-firmware.s3.amazonaws.com/rg1xx-lora-gateway/firmware/newest/fw.txt 

After this the laird is updated but settings are cleared. So you need to reconfigure it. In this laird sentrius RG1xx post you will find the needed information for login into it. By doing this operation, at a first step I stayed in TTNv2. You may know that n upgrade from 93.7.x to 93.8.x will force you to re-create a new gateway in backend (naming are different and laird admin is not letting you select the ID)

In TTNv2, the Basic station implementation is limited and the gateway does not need to have a dedicated Key. To switch from Semtech legacy protocol to Basic Station, you need to go to Forwarder menu and fill the LNS server address, then select the certificate (see above table), Upload Certificate then click on Update button. there is nothing more to do as the gateway EUI is the same so the TTN backend will see it the same way.

Upgrading and configuring MikroTik LoRa8

LoRa8 can be upgraded to version 6.48.1 as of today. the version you are running can be seen in the menu System >> Packages. Then you can click on Check for Updates and execute the update.

This gateway does not supports Basic Station so you will have to use Semtech Legacy protocol. The configuration of this gateway is documented, for TTNv2 on one of my previous post about Mikrotik LoRa8 gateway and on TTNv3 in the official documentation.