There are some domains where web3 application are promising. I already explored storj solution to store data and as a user point of view, it has convinced me. Now, I’m exploring the capabilities proposed by Streamr Network. This project has started in 2017 and still improving its development roadmap. It proposed publish & subscribe solution for standard protocols like MQTT, HTTP, Websockets. By design the solution is scalable and reliable. It currently has more than 4000 online nodes. The project will also propose a marketplace where to purchase data by subscribing to flows.

This blog post will detail my experience with this project and will be updated all along my usage of it, as miner and also as user.

Tokenomics

The DATA token is based on ERC-20, a previous token is now named XDATA, value on the market is currently about $0.03 in August 2022.

The DATA token is used for securing flows. Securing flows, means add more actors to relay the flow and provide a better reliability. For this, someone needing a reliable flow will deploy found in a BOUNTY. Someone who is doing this is named a SPONSOR.

When BOUNTIES are created, the BROKER NODES select the bounty they want to work for. They need to stake DATA as a way to make sure they will offer the service for the expected time duration. If they are not correctly supporting the service, the staked token will be slashed. By providing the service, they will get the DATA token of the bounty. Multiple Broker can work on the same bounty and the share of reward will be based on the amount of stacked DATA.

There is a last role : the DELEGATOR, this is basically someone that do not want to run a broker but want to earn DATA from the network. This persona gives stack DATA for a given broker and will earn a share of the broker revenues.

My opinion is that this model is similar to chia (storage) tokenomics, there are some point I don’t really like:

• Usage is payed with DATA, you have no predictability of your usage cost
• Broker revenue can be negative even if you do your best, due to the network and software stability. It creates a market for people bidding on your incapacity to provide the services.

These are in my point of view important limitation in the development of the solution usage.

Mine by becoming a Broker Node

Setup the broker

[root ~]# mkdir /streamr
[root ~]# docker run -it -v /streamr:/home/streamr/.streamr streamr/broker-node:latest bin/config-wizard
? Do you want to generate a new Ethereum private key or import an existing one? Generate
? We strongly recommend backing up your private key. It will be written into the config file, but would you also like to see this se
nsitive information on screen now? Yes
: This is your node's private key: 0xxxxxxxx
? Select the plugins to enable websocket, mqtt, publishHttp
? Provide a port for the websocket Plugin [Enter for default: 7170] 7170
? Provide a port for the mqtt Plugin [Enter for default: 1883] 1883
? Provide a port for the publishHttp Plugin [Enter for default: 7171] 7171
? Do you want to participate in mining and staking? Yes
? Select a path to store the generated config in /root/.streamr/config/default.json
Welcome to the Streamr Network
Your node's generated name is xxxx

During the setup phase, different question are asked,nothing complex to answer at all.

This will create a Ethereum Wallet, so make sure you saved the private key or you will never be able to get your token back after this.

After this you need to open your network ports 1883,7170,7171 for giving access to these services.

The config directory must be accessible to the docker user.

Run the miner

I’m using the following script to run the broker as a docker container:

#!/bin/bash
docker pull streamr/broker-node:latest
docker stop streamr
docker rm streamr
docker run -d --restart always \
        -p 7170:7170 -p 7171:7171 -p 1883:1883 \
        -v /streamr:/home/streamr/.streamr \
        --name streamr \
        streamr/broker-node:latest

You can check the logs with docker logs -f streamr

In the log you will see the explorer url where to monitor your broker (like this one randomly taken)

Stack to earn token

Currently all the described tokenomics previously described is not in place and to make it simple the only way to mine token is to stack token on your broker node. The maximum stacking quantity per broker is 10.000 tokens. You can have multiple broker, even on the same IP address, up to 5 if I understood well.

You need to transfer some DATA token to your Broker node on Polygon network. From Binance you can directly send DATA on Polygon so it sounds quite easy. Buy some DATA token with Binance and withdraw them with Polygon network to the node address. I recommend to do a try with a small amount of DATA to get started and make sure you use the right addresses.

To get your node address on Ethereum you need to take a look at the docker container logs:

# get your Ethereum Wallet ID
[root ~]# docker logs streamr | grep Ether
INFO (broker              ): Ethereum address 0x2BEdxxxxxxxxxxx 

# get your Node ID
[root ~]# docker logs streamr | grep "Network node"
INFO [2022-08-22T13:03:37.362] (broker              ): Network node 0xxxxxxxxxxx running

The transaction seems to have a flat cost whatever the number of DATA transferred (for my tests 5 DATA) and takes about 10 minutes for the validation.

One you have transferred some DATA token, you can watch it:

• ETH wallet – https://polygonscan.com/address/0x3xxxxxxxxxxxx
• Node data – https://brubeckscan.app/scan then search by Ethereum adress
• Node – https://brubeck1.streamr.network:3013/datarewards/:nodeId

Get your rewards

The rewards are added to your stacking account on every 1st day of the month. You can move your DATA back to Binance or anywhere else. But to access your DATA wallet it is a bit tricky…

DATA token are on polygon smart-chain, running on Ethereum and the best way to access it is to use Metamask, and you need to make most of the settings manually. Once you have installed Meta-mask, if you don’t have polygon in the network list, you need to add the Polygon Network the parameters are the following:

• Name : Polygon
• RPC Url : https://polygon-rpc.com
• Chain ID : 137
• Token Symbol : MATIC
• Explorer : https://polygonscan.com/

Once you have the polygon network you can import your wallet with the private key you previously saved (when you did the streamr setup). For this in Metamask, click on the account selection and select “import account”, then copy/paste your private key.

You may see your account, make sure you are on polygon main net on top of the screen and your wallet ID under the account icon is corresponding to your public Ethereum address (see above).

Now, you need to click on import token and select custom token. The custom token is:

• Address: 0x3a9A81d576d83FF21f26f325066054540720fC34
• Symbol: DATA
• Token decimal: 18

Once done, you should see on Metamask your token balance and being able to swap it or transfer it.

How to use the messaging service

Streamr proposes a messaging solution, you can create streams, I’ll show how to create mqtt stream. As an IoT solution provider, MQTT is a standard for exposing data.

It seems that messages can also be stored in storage node. I do not actually know how the storage part works. Protocol also supports message signature and payload encryption. Streams also support partitioning for the scalability. Most of these options are not standard for MQTT ; Streamr proposes some interesting features.

Streams can be provided in a marketplace with a given access price for a given period of time. The monetization aspects is interesting even if given in DATA token. Using the crypto token can be a long term problem when the token value is growing.

Create a stream

To create streams, you need to use the Streamr core web application. This application requires a wallet to be identified. You can connect using MetaMask or WalletConnect. I honestly think that is also a limiter to the adoption. Like using the token as a payment way for using the solution, it means companies will need a lot of time and internal transformation to be able to manage such type of payment. the eco-system will need to have some bridge services where to buy streams in fiat for having a growth on usage. The Streamr client allows to make the needed automation for this.

So basically you need to use MetaMask to access the messaging service as a publisher. You can create a new account or use an existing one. In my point of view it’s better to use a different account for publishing streams and for your broker. Here is what you need to do for your initial setup:

• Install MetaMask plugin on your browser
• Create a new wallet, save your words…
• Get some MATIC token (10 for about $8) on Binance or anywhere else. MATIC will be needed for paying gas during the stream creation process.
• Transfer these MATIC to your created wallet using Polygon network.
• Switch your wallet network to Polygon.

The stream architecture is simplified compared to what you can do with MQTT. As a consequence you won’t be able to migrate existing solution using MQTT into Streamr. This point is a bit annoying as it means you need to design your integration to match with Streamr design. A flow is only identified by a domain (basically your Etherum address) and a path name. The path name can have multiple path like /device/temperature. But this is a constant value.

Usually when you design solutions with MQTT you have stream named like this: application_name/device_id/temperature it means, for the device 1234 in the application tempApp: tempApp/1234/temperature. MQTT allows a dynamic creation of flows. With StreamR it seems complicated to have a such dynamic creation. Flows definition is static. The CLI allow to dynamically create flow, so it can be automated but not free.

That said, you can configure your broker, to remove the domainId in the configuration file, that way you can start your flow name directly with a standard structure.

plugins: {
    "mqtt": {
        ...
        "streamIdDomain": "0x1234567890123456789012345678901234567890"  // or "mydomain.eth"
    }
}

With a such setting your stream path can be application_name/device_id/temperature as with a standard MQTT broker. (but the stream creation is still static or programmatic).

You can document your flows by defining the field structure, it seems to work with single dimension payload only. I really no recommend to use “detect field” button as it has been created 600+ entries I had to removed manually.

When you click on Streamr creation, a Metamask popup is requesting to validate the transaction, the stream creation cost has been 0.016 MATIC ( about $0.01 ). The flow is private by default and you will be able to share it later.

Every flow modification will generate a new transaction and have a cost, like switching a flow as public cost me 0.003 MATIC (quite nothing)

Using ENS domain

Instead of using the 0x123456479.. address to name your flow, you can register and ENS domain. It basically works like a domain name. You can register it on https://ens.domains/ so your flow will become more simple to share, like disk91.eth as a domain name registration this is not free. Instead of this it is a bit expensive in my point of view as the price is equivalent to classical domain name cost range in a period of time where ETH is low : 0.003 ETH per name + gas fees or a single year. For making this you also need another crypto currency : ETH. Add some more to set you domain as default.

I did not find the way to update a previously built stream from the address name to ens name, so better register your ens name before creating the flows. So you can select it at the creation step.

Setup your stream access right

To publish your stream on the network, you need to Share the stream with the broker you want to use (if you are not using the same account to manage your broker and publish your streams)

For this you can go in the Share tab of the flow and add your broker address with the Publish and Subscribe roles.

If you want anyone able to read that stream, you need to makes it public.

Publish messages on streamr

Once the flow exists, you can publish messages on that flow. For this you can deploy a light node (Node.js) or target your broker. Let’s try to directly connect to the Streamr broker. We need to get the APIKey to connect to it. This one is located in the Streamr broker configuration file:

"apiAuthentication": {
        "keys": [
            "UIDHUSHIOS76767Z9H7ZHD8H9"
        ]
}

Once you have it you can publish a message (make sure you have a correct json or it does not work out of local mqtt broker, use ” not ‘:

[root ~] mosquitto_pub -h streamr.disk91.com \
  -t '0x3b51a16e47d8921ebd767a77e280949fcda2ffa3/myApplication/testStream/temperature' \
  -i dsk-1 -m "{ \"temp\" : 23 }" \
  -q 2 -p 1883 -u 'anyThing' \
  -P 'UIDHUSHIOS76767Z9H7ZHD8H9'

And you can get the result from the broker:

[root ~] mosquitto_sub -h streamr.disk91.com \
-t '0x3b51a16e47d8921ebd767a77e280949fcda2ffa3/myApplication/testStream/temperature' \
  -i dsk-2 -u 'anyThing' \
  -P 'UIDHUSHIOS76767Z9H7ZHD8H9'

{ "temp" : 23 }

In this mode the communication is going to a single broker and does not get benefit of any decentralization. Reader must know the broker API key so it is quite limited as you need to know it to read the messages. So for this you can use the SDK with some piece of JS code:

For a quick POC you can create a docker image with the needed components with the following Dockerfile and other file (this can be much more simplified, it is a quick & dirty approach):

## Docker file

# syntax=docker/dockerfile:1
ARG SYSTEM_TIMEZONE="Europe/London"
FROM node:latest
WORKDIR /usr/src/app
RUN apt-get update && apt-get install -y vim
COPY package*.json ./
RUN npm install
RUN npm install streamr-client
COPY . .

EXPOSE 8080
CMD [ "node", "server.js" ]

## package.json
{
  "name": "docker_web_app",
  "version": "1.0.0",
  "description": "Node.js on Docker",
  "author": "First Last <first.last@example.com>",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.16.1"
  }
}

## server.js
'use strict';
const express = require('express');
// Constants
const PORT = 8080;
const HOST = '0.0.0.0';
// App
const app = express();
app.get('/', (req, res) => {
  res.send('Hello World');
});
app.listen(PORT, HOST);

Then you can build and start the container:

[root ~] docker build -t streamr-cli .
[root ~] docker run --rm -d --name streamr-clic streamr-cli
[root ~] docker exec -ti streamr-clic /bin/bash

Once in you can create a subscriber (or anything else) and run it.

## sub.js File
const StreamrClient = require('streamr-client')
const streamr = new StreamrClient({
    auth: {
        privateKey: 'provide your Eth account private key'
    }
})

streamr.subscribe('disk91.eth/helium/mapping', (msg) => {
    // Handle incoming messages
    console.log(msg);
})

## to run it
[stearm-cli] node ./sub.js

Access messages and stream information publicly

You can take a look at your flow and the associated architecture on the explorer. Here is my flow with storage activated in Germany.

You can also access your stream on the stream explorer when the stream is public, the address is https://streamr.network/core/streams/disk91.eth%2Fhelium%2Fmapping/preview with the name of the stream to access in the middle of the URL. (it would be great to have a page where to type the path and show the stream to makes it more user friendly for simple user. When you are the owner of the flow you have a button in the stream details to directly access this page.

Create a first use-case to use streamr

You know I’m making mapping device for helium network and computing information on backend with the project WioLoRaWanFieldTester and now some derivatives with other brands. This is currently a fleet over 500+ active mappers constantly reporting coverage information to helium mapper, coveragemap and cargo systems. But these information were not available publicly to let anyone else study them or make other mapping services.

The goal is to use Streamr to distribute this information (noting better than using web3 to run web3…) and makes it accessible to anyone. For privacy reason, the device ID is not published in the flow.

So, for this i’ve been made the following stream : disk91.eth/helium/mapping it is publicly accessible on https://streamr.network/core/streams/disk91.eth%2Fhelium%2Fmapping/preview

You can subscribe to the flow with the CLI code above. You get the following kind of data:

{
  latitude: 42.8168213,
  longitude: -1.7529057,
  altitude: 381,
  sats: 15,
  hdop: 70,
  meta: [
    {
      rssi: -70,
      snr: 6.199999809265137,
      hotspotId: '1125Y8DPRFEo5kdU5zwuATNdDi8HCqp3DjUP4KJHByREd6vEgpdQ',
      hotspotName: 'acrobatic-cotton-tadpole',
      dist: 83.16029590079458,
      lat: 42.8175006250219,
      lng: -1.7524792756234515
    },
    {
      rssi: -120,
      snr: -15.199999809265137,
      hotspotId: '112P5c3QocvkhWhFivGh6KmFuS6FGwZVr7XUJ2Xs6qynRCicKJgq',
      hotspotName: 'shallow-concrete-lemur',
      dist: 6532.19687075256,
      lat: 42.83745195330477,
      lng: -1.6779083104695316
    },
    ...
  ]
}

You can access this flow as a product in the streamr market place