From 9-12 months ago, all the backend Sigfox user have this notice printed in the callback page:

Since, I hear many people saying the duplicates and signal related metadata will be removed from Sigfox.

This a WRONG ! Let see exactly why and what is replacing this feature ; how it impacts the way you build you integration with Sigfox backend.

What are signal related metadata

The metadata information are sometime as important as the data itself. As part of the Sigfox metadata you can find device ID, message time stamp. As well you can find information about radio signal: the signal strength (RSSI), the signal over noise ratio (SNR), the Sigfox station receiving this signal.

You can choose to ignore these information or process them. In my point of view, ignoring these information is an error for the following reasons:

• The number of station and the signal quality gives you a good idea of the reception quality and the risk of losing frame.
• The station identification let you know if the device is mobile or fixed, eventually gives you information on devices on the same area. It can also be used to position a device.

These information are really important for device management, a critical peace of any IoT service.

How did it worked ?

Basically the station metadata were provided as part of the DATA callback. So for each of the received messages from any of the station you had a callback (web hook) contacting the customer backend server to provide these information.

As a consequence if your device was in a dense area for one message you had about 10-20 callback on customer backend. Let’s imagine you have a fleet of 100,000 devices communicating 50 times a day, this is about 50-100M callback executed per day. This is up to 1150 callback per seconds. I let you imagine the Sigfox outgoing traffic per seconds…

So the way it was implemented was not efficient.Tis modifhis is why Sigfox has decided to change the way it works by removing from the data callback the station related metadata.

How will it work ?

People have thought, Sigfox will remove this feature and it has been a kind of panic during a couple of weeks due to a lack of communication on the future of this feature. The communication still not totally clear but the information on the documentation has been published and I’ve been able to switch some of my backend to the new system.

So, the new way to access these information will be to use a new DATA_ADVANCED callback. This callback is fired about 30 seconds after the DATA callback. This delay is to ensure all the stations have reported frame reception.

The DATA_ADVANCED callback contains the data and all the associated meta-data: position, duplicates (station & signal) as new metadata: operator & country code. In one callback you will have all the information related to one transmission instead of having one per station with a lot of redundancy.

What does it looks like ?

The DATA_ADVANCED callback can only transport the duplicates & computed location as part of the body of the frame. The use of POST/PUT method is mandatory.

The content of the body can be something like this:

{
  "device" : "{device}",
  "time" : "{time}",
  "data" : "{data}",
  "seqNumber" : "{seqNumber}",
  "lqi" : "{lqi}",
  "lat" : {fixedLat},
  "long": {fixedLng},
  "operatorName" : "{operatorName}",
  "countryCode": {countryCode},
  "computedLocation" : {computedLocation},
  "duplicates" : {duplicates}
}

You can notice some new fields:

• lqi : Link quality – a plain text string like “Excellent”
• fixedLat, fixedLng – is the position you manually set to the device. There is nothing computed or automatically added here. So basically you have 0.0 when nothing has been setup.
• operatorName – a plain text string like “SIGFOX_France”
• countryCode – the country code according to ISO 3166-1 numeric norm.

The content of the duplicates and computedLocation is also defined by Sigfox itself. The JSON format has been chosen and you can’t modify it. This is something new and in my point of view it not really elegant. I’m not sure many people still use XML but if you used it, now it’s finished. So even if you did not want to deal with JSON, you will have to.

A duplicates structure is following this structure:

[
  {"bsId":"0DC0","rssi":-121.0,"nbRep":3,"snr":25.99},
  {"bsId":"0AE1","rssi":-116.0,"nbRep":3,"snr":27.84}
  ...
]

• bsId – is the station ID
• rssi – the rssi value as a double
• nbRep – the number of replica received by this station
• snr – the signal on noise value ( maximum value usually 30.0 )

Before being able to access the duplicates information you need to request a specific right from Sigfox support. So if you try to activate duplicates in DATA_ADVANCED you will get an error until you create a ticket to support and get it validated.

important notice !

The computedLocation structure is also a JSON pre-defined structure.

{
  "lat":23.783814666368414,
  "lng":2.1434994169512827,
  "radius":8869,
  "source":2,
  "status":1
}

This structure can be added to the DATA_ADVANCED callback when you have a Atlas location option activated to your contract, or the WiFi location option activated. You can’t get it with if none of these options are activated on your contract.

• source field is indicating the source used to compute the location (atlas, WiFi, GPS decoding)
• status field is set to 1 when a position has been computed. 0 when no position has been computed. In this case only the “status” field is returned with value 0. Nothing more.

What does it change ?

Basically this doesn’t have a direct impact on your previous backend code but you will have to rewrite your callback handler.

However, it can impact the way you manage your integration with Sigfox backend if you want to take the full benefits of this new feature:

Upload only devices

For devices with no downlink you can choose to only use DATA_ADVANCED callback instead of the combination of the two. That way you will have 1 callback with all the information in it, including data. Donwlink is not working with DATA_ADVANCED as the 30 second delay are out of the Sigfox downlink timing.

Backup integration

In my post about how to interface with Sigfox backend I detailed how to use a backup server to ensure you miss no data from callbacks. This was consuming the DATA messages with duplicates. Now you can do it only using the DATA_ADVANCED message as the 30 seconds delay is not a problem. Less communication you will have, better it will be for the backup server.

When will it be available ?

This is already available from backend, including the duplicates as soon as you request access to it. From the Sigfox backend website the deadline to be in production with this new callback is still 1st of June 2019, date of the previous solution deprecation.

That said, I’m not fully sure the previous callback will really be deprecated on June 1st…

And this is not a reason to not migrate ! Just do it, this will consume less energy and save penguins.

Update: the official communication about postponed deprecation