Storj is a blockchain in competition with cloud providers like Amazon to offer a decentralized data storage managed over a blockchain. This is the kind of project I like as they are running blockchain to offer tangible services in the real world and competing with existing business in a new way.
I previously did some experimentation with Sia proposing the same thing but I’ve been really disappointed by the way it works. I’ll detail it later in the blog post. So I decide to create a Storj node to offer storage and also try using Storj to store some of my data.
Storj have an advantage compared to cloud provider, the warranty of a high level of protection of your personal data. The data is spread over multiple nodes and strongly encrypted. In term of price, I’ll detail it also later in the blog post.
How does Storj works ?
What is storj and how can it be used ?
Storj is a decentralized, secured, muti-region data storage offering a standard S3 interface. As a consequence, it can be use with many backup & storage solutions using this technology.
The price for using Storj as a customer looking for data storage is free up to 150GB then $4 per TB stored. The bandwidth is also billable at $7 / TB over the free 150GB. As a comparison a standard price for Amazon S3 is about $23 / TB stored and $90 / TB transferred.
The Amazon low cost storage, Glacier, cost about $4 / TB + $90 / TB transferred but you need to wait up to 12 hour for a restore. Storj is a S3 level of service for the price of Glacier with really lower bandwidth costs. Only Glacier Deep Archive is competing to Storj with a price of $0.9 / TB stored if you keep it for up to 20 months, otherwise the transfer costs makes it less interesting than Storj. All these comparisons are macro as S3/Glacier billing is really more complex. They are just to give a comparison idea between Storj market proposal.
You can use Storj to backup your data, it is compatible with the AWS S3 protocol (when using the S3 gateway) so you can use it with many existing backup solutions. You can also use the Storj native protocol and the associated CLI.
For a basic usage, you can also drag & drop your files to Storj using the dashboard.
Storj network statistics
You can access the network statistics to see the number of nodes and related payments on the storjnet.info website.
How to propose storage and get rewards ?
Before getting data, a node needs to get a reputation and when you join the network, you need to wait a certain time (about 6-8 months) before being eligible to receive data and payout. This can be a bit long but the investment to join the network is low as the required hardware is basically a raspberry pi associated to an external storage. You can setup a system around $200 with 5-6TB.
I’m not yet familiar with the rewards in this solution but assuming the indicated price above are mostly given to host with a certain redundancy, it could be around $30 a month. Apparently the network traffic is more profitable than the data storage itself.
It seems that the payment equivalent is about:
- $1,5 per TB stored
- $15 per TB transferred, assuming, 20% of total stored as transferred ratio.
Storj Payment, are made on STORJ token, not in $, you also buy storage in STORJ or in $ converted in STORJ by the company behind Storj project. Conversion price is based on token market price. There is no mining in Storj project : all the Storj have already been mined (425M). So Storj project basically uses the token to pay the storage node on monthly basis. As a consequence, storage has no cost for the project itself and it can support the growth until the solution is adopted by market and real money comes to give a tangible value to the token.
If all the token has been mined, it does not mean they are on the market. Storj have a system of stacking where the token are lock-up during a certain time period. Storj project can, at the end of the period decide to lock them up again or use a part of it.
Basically, if Storj usage is low, the project needs to get more token to pay the nodes, if the usage is high, token have been returned to project and more token can be locked. According to coinmarketcap, the circulating supply is currently 319M on the 425M.
The main risk on this project in my point of view is if the equilibrium is not obtained before the use of all available tokens because it means there will be no more token to pay nodes. This is also directly related to the market price.
On top of the revenues related to storage services, the network is withholding a part of the tokens to secure the way you exit the network. Basically, all along the time a share of the revenue is locked and will be redistributed on graceful exit. One of the related problem is a disk crash will be a bad exit and you will loose the corresponding locked tokens. The lock works the following way:
- Months 1 to 3 – 75 % of Node revenue is withheld, 25% paid
- Months 4 to 6 – 50 % / 50 %
- Months 7 to 9 – 25 % / 75 %
- Months 10 to 15 – 100 % is paid.
- Months 16+ – 50 % of total withholding are returned, remaining 50% held
A good recommendation is to graceful exit before crashing ! Why a such thing ? easy to understand : when badly exiting, the redundancy of data needs to be repaired, this means data flow to be paid by the network to the other nodes. This is covering these costs.
Typical setup to get started
For my setup, I’ve selected the following items on Amazon (I do not recommend this setup)
- 5TB external drive from Seagate ($120)
- Raspebrry Pi3 B+ ($49,9)
- Raspberry Pi3 B+ casing & passive cooling ($16)
- Raspberry Pi3 power supply ($15)
- SD Card 32GB ($10)
The total price for 5TB is $211
Currently the storage price is really high due to shortage in hard-drive. I hope to increase the storage later with a second hard drive offering a larger capacity for a lower price. I’ll let you know.
For this reason, I’ll install my hard drive in a way I can later extend it easily.
You can also use solutions based on NAS, apparently Synology and Qnap works well with Storj if your have a Nas version with a “+” allowing to run containers on it. This is a good way to start with Storj with no investment if you have a NAS with available free storage.
After one year, I do not recommend the previous listed setup: I had too much instability with the raspberry Pi and the external drive running on USB. At the end, I earned nothing as my nodes have been disqualified. I’m currently a different setup based on a Mini-ITX computer with the previous storage and able to support different blockchain decentralized infra projects. I’ll update that blog post based on future results.
Return on invest estimation
This is really a rough estimate of the return on invest with Storj. There is many errors in this calculation: first, I don’t know all the exact Storj mechanism and use personal estimation. Then the Storj price evolution have to be taken into consideration. Then, the local regulation and taxes also need to be taken into account. Also, the storage price is really high currently and this is impacting the initial investment. For sure, Storj is a long term investment. In green on the figure above the turning point, between 18-24 month. All numbers are 6 month total.
Currently, I never achieved to get a single $ due to setup instability.
Get started – Install Storj
At first, you need to have an invitation code. For this, register on https://www.storj.io/node enter en email address and you get a code to be used later when creating your identity.
I installed Storj on a Raspberry Pi 4 with an external drive with 3TB of space following the documentation available on Storj website.
Key generation add-on
The key generation process is quite long on a raspberry Pi even v4, so it is recommended to docker psgenerate it on a desktop. As a comparison, I had to generate 350M keys to get one with the expected complexity and this took about 2 hours on my Mac mini 2019. (The Pi is about 20M keys generated per hours).
Hard drive configuration for later extension add-on
The disk configuration can be made different ways, the one indicated in the Storj documentation is the basic one. If you want to be able to later increase your storage by adding some new hard-drive, it could be a better idea to use a raid0 (stripping) configuration.
Make sure you have and empty disk with no partitions, then:
# install lvm
[~] sudo apt-get install lvm2
# purge /dev/sdX
[~] sudo wipefs -a /dev/sdX
# configure /dev/sdX and volume
[~] sudo pvcreate /dev/sdX
[~] sudo vgcreate storj_vg /dev/sdX
[~] sudo lvcreate --stripes 1 --stripesize 4k -l 100%FREE -n storj_lv storj_vg
# format
[~] mkfs.ext4 /dev/storj_vg/storj_lv
Get the device UUID from created fs, this is printed on the mkfs.ext4 output, you can also get it from:
[~] blkid /dev/dm-0
/dev/dm-0: UUID="9a5324db-743c-42f2-9d9f-4ac81bde8bdc" TYPE="ext4"
Then you can add the partition in /etc/fstab file, after creating the /mnt/storj directory
UUID="9a5324db-743c-42f2-9d9f-4ac81bde8bdc" /mnt/storj ext4 defaults,nofail 0 2
Dynamic DNS ADD-ON
You also need to have a dynamic DNS if your IP address is not public, this is required later when starting the container. On Raspbian, you have ddclient package to make this easy.
[~] sudo apt-get install ddclient
# I chosen to use noip.com config is
- Server : dynupdate.no-ip.com
# you may have to change the configuration file at
# /etc/ddclient.conf
daemon=5m
protocol=noip
use=web, web=checkip.dyndns.org
server=dynupdate.no-ip.com
login=<noip login>
password='<noip password>'
<your ddns entry like xxx.ddns.net>
# make sure ddclient is starting at boot
[~] sudo systemctl enable ddclient
#make sure it is running
[~] sudo systemctl status ddclient
# you can manually test ddclient with
[~] ddclient -daemon=0 -debug -verbose -noquiet 2 /etc/ddclient.conf
Storj start script add-on
During the last step of the installation, make sure your execute the docker setup command once.
docker run --rm -e SETUP="true" \
--user $(id -u):$(id -g) \
--mount type=bind,source="<identity-dir>",destination=/app/identity \
--mount type=bind,source="<storage-dir>",destination=/app/config \
--name storagenode storjlabs/storagenode:latest
Then you can launch the docker command for the run.
here is my run.sh script to (re)launch storj
#!/bin/bash
docker pull storjlabs/storagenode:latest
docker stop storagenode
docker rm storagenode
docker run -d --restart always --stop-timeout 300 \
-p 28967:28967/tcp \
-p 28967:28967/udp \
-p <my_local_ip>:14002:14002 \
-e WALLET="0xb451BE5xxxxxx" \
-e EMAIL="foo@bar.com" \
-e ADDRESS="<my_public_ip>:28967" \
-e STORAGE="xTB" \
--log-opt max-size=50m \
--log-opt max-file=10 \
--mount type=bind,source="/mnt/storj/storagenode/identity/",destination=/app/identity \
--mount type=bind,source="/mnt/storj/storagenode/",destination=/app/config \
--name storagenode storjlabs/storagenode:latest
Storj container needs to be updated on regular basis, you can check the last version of Storj on https://version.storj.io/ you can see the minimal version required and the suggested one. Sometime previous version can be still accepted for a certain time.
Here is the script I’ve made to check for manual update:
#!/bin/bash
if docker pull storjlabs/storagenode:latest | grep "Downloaded newer image" ; then
# Image updated, call the previous script that restart everything
./run.sh
fi
It sounds possible to cron that script to update automatically (be careful with the related path used for the subscript) but the right way for automatic update is to use the following way:
It seems that a better way to do it is to use watchtower docker image. It do the job. Unfortunately when using it on PI, I’m getting a warning about the platform, according to my recent test, it seems working correctly even with this warning.
Remote access
You will need to upgrade the Storj software on regular basis and as a consequence access you node remotely (if not deployed at home)
To be able to remotely access the node for making the updates, I’ve used the sshtunnel technic documented in this excellent blog post.
On your storj node, create a key to identify it
[~] sudo mkdir -p /etc/sshtunnel
[~] sudo ssh-keygen -qN "" -f /etc/sshtunnel/id_rsa
Copy the id_rsa.pub content into the authorized_keys on the remote machine.
Then create a systemd service to automatically create a ssh tunnel on an external server where you will be able to open “locally” a session to your node. This file is saved on /etc/systemd/system/sshtunnel.service
[Unit]
Description=Service to maintain an ssh reverse tunnel
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=0
[Service]
Type=simple
ExecStart=/usr/bin/ssh -qNn \
-o ServerAliveInterval=30 \
-o ServerAliveCountMax=3 \
-o ExitOnForwardFailure=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-i /etc/sshtunnel/id_rsa \
-R <localport_on_your_remote_server>:localhost:22 \
sshtunnel@<your_remote_server>.com -p 22
Restart=always
RestartSec=60
[Install]
WantedBy=multi-user.target
Then you need to create a sshtunnel user on the remote machine and add the public key created to the authorized_keys file of this user (read the linked blog post for details)
Then the sshtunnel service is started
[~] sudo systemctl enable sshtunnel
[~] sudo systemctl start sshtunnel
# verify
[~] sudo systemctl status sshtunnel
To connect on the storj node, connect to your remote_server, then use:
[~] ssh -p <localport_on_your_remote_server> pi@localhost
Monitor your storj node
Monitoring the storj state
Once launched, you can access the dashboard page on the local IP, port 14002
On the raspberry Pi, you can monitor the node state with the following command (Quit with CRTL+C):
[~] docker exec -it storagenode ./dashboard.sh
You can also take a look on the global Storj statistics on https://storjstats.info
Monitoring your PI instance
It is also important to monitor the raspberry pi as any stop will decrease your reputation and your chance to host data. The uptime is really important. It is considered that the maximum downtime in a month is 5 hours.
For the monitoring, I was using a free service NodeQuery it is really easy to deploy on the pi following the process described when adding a server and it sends you alarms in case of host vi failure. You can setup 10 host with the free account. Unfortunately, this service seems to be out of support and now down. I’ve changed for Netdata cloud. This service is also free and really powerful, as a consequence it is also more impacting you system and network.I recommend to use the docker version, more easier to deploy.
Use Storj
You can be a Storj user without being a storage provider. This is really simple to use if you already know object storage like S3. The principle is basically to store file (objects) in containers (directories) to make it simple.
You need, at first, to create an identity and keys, for this you registrer on https://eu1.storj.io/signup (adapt the zone to yours). Once done you can access the dashboard.
Here you can use the drag & drop interface to store and retrieve files. This is user friendly for storing a couple of files.
You can also create access grants to be used by external clients. This solution is better for ensuring the end-to-end encryption of the objets transferred. You can use, as an exemple the CLI tools well documented.
I’m currently using Storj to externalize some of my backups, as part of the free 50GB storage, the backup speed is really good from my server and the setup has been easy. Previously I was using Amazon S3 service for this and my script has been adapted really quickly as the CLIs are really similar.
You can upgrade the 50GB free storage to more (apparently 150GB with different steps) but the objective is to have you going to a payed storage … this is a need to make the project relevant 😉
Competitors
Storj is not the only blockchain using storage. On of the well known project is Chia, but this one, if it consumes a lot of hardware and is a part of the cause of the really high storage price today, it is not providing storage space as much as I understand it.
Sia is the direct competitor I know, I did not write a blog post about it, even if I used it for a couple of weeks because I did not find it really relevant. Here is why.
Sia is basically the same as Storj but it manages the node confidence differently. Instead of testing the nodes during 6 months, it directly trust the nodes with a rewarding and loss mechanism for nodes. Basically, when you are a node, you need to have Sia in your wallets and every time someone is renting storage on your node, you block a part of your wallet coin as a compensation in case you won’t fulfill the storage contract. In shorter works : if your node goes down during the contract period of time (couple of month) you will loose that amount of money. This amount is basically higher than what you can get from the contract. Honestly I did find the equilibrium for the node in this and basically I lost money by stopping running that project. The mechanism behind are complex for node host and also for users.
The worst part for Sia in my point of view is that all the costs are in Sia, and contract are long term, so basically, you don’t really know what will be your cost and earning to store and host data. Using Sia in a long term perspective is a high risk.
This project could be cool, as the interfaces are well done and it is easy to getting started with it but in an economical point of view, in my opinion it is too much complex and to incertain in terms of costs and revenue.
Conclusion on Storj
In this category of blockchain project to store data in a decentralized way, Storj seems, in my point of view the most mature solution. The return on invest is a bit long particularly in a period where the storage price is high. The first 6 month period also create a certain insecurity but the hardware involved in the project can be used for many other purposes in case of abandon. So the risk level on the investment seems acceptable.
I think the best way is to start with a reduced investment time to get your node a good score and then you can upgrade your node with more storage, expecting a lower hard drive cost in 6 months.
For sure, the project is mature enough to be used to store data, the main risk is at anytime the project stops (like if the Storj token are all distributed), in this case the stored data can be lost quickly. So you need to have a certain level of trust to start on it and regularly follow the project health like the number of active nodes, total storage…
The number of active node can be checked on http://storjnet.info/ page. The total available space is not an information I found on the different dashboard I found unfortunately.
If there is one thing missing on that project is clean and powerful dashboard with nice UI…