Pair the miners (4/6)

This tutorial is part of a series of articles introduced here.

In part 3, we configured the miners of our private blockchain.

NB: It is important that what we mean by “private Ethereum blockchain” in this context has absolutely nothing to do with the “private blockchain” championed by Hyperledger, Eris/Monax, or the recently announced Enterprise Ethereum Alliance. These are different codebases with different client applications, so they correspond to different implementations of a different set of concepts. They are private in the sense that they limit who has access to their chain. They are also called permissioned blockchains, and to be perfectly transparent, we wish they were not even called blockchains at all. In this series of articles, what we call “private blockchain” is a private instance of the Ethereum implementation. Same code base, same client application, different network identifier and genesis block. In other words, what we will come to create in this series is a chain with the same rules as the main chain, the same consensus algorithm too, but a different root block. This will become clearer in part 3. For more information about the 3-layer model and differences between concepts, implementations and instances, you can also watch our Devoxx talk and read this article.

We saw that transactions are not propagated from miner #1 to miner #2.

Remember that a blockchain is a peer-to-peer network. This means that our miners need to see each other in order to propagate transactions.

In addition, the discovery protocol is not working on a private blockchain. This means that we have to configure each node to specify the identity and the location of its peers.

This part describes how to pair our miners.

Let’s start by retrieving the node information that uniquely identify each node deployed in the blockchain.

Step 1 Clean your miners

We noticed some issues when we tried to synchronise our miners having several thousand blocks. Sometimes, miners were not able to pair or the synchronization process became too slow.

The easiest way is to reset the chaindata of the private blockchain installed on each miner.

Step 1.1 – Stop miners

First ensure that your miners are stopped.

As a reminder, you can either press ^C on the miner’s console, or search and kill the “geth” process:

computer$ ps aux | grep geth

eloudsa 43872 0.0 2.2 556717632 363492 s001 S+ 3:49PM 1:58.01 geth --identity miner1 --dev

computer$ kill -INT 43872

Step 1.2 – Delete the chaindata

Delete chain data for miners:

computer$ rm -rf ~/ChainSkills/miner1/geth
computer$ rm -rf ~/ChainSkills/miner2/geth

 

 Step 1.3 – Initialize miners

We initialize our miners with the genesis block.

Start with the miner #1:

computer$ cd ~/ChainSkills
computer$ geth --datadir ~/ChainSkills/miner1 init genesis.json

Then the miner #2:

computer$ cd ~/ChainSkills
computer$ geth --datadir ~/ChainSkills/miner2 init genesis.json

Step 2 Get IP address

Get the IP address of your computer running miners.

computer$ ipconfig getifaddr en0
192.168.1.39

Replace the interface according to your network settings:

  • en0: wired/ethernet
  • en1: wireless

Step 3 Get Node info from miner #1

Step 3.1 Node info for miner #1

Let’s start the miner #1:

computer$ cd ~/ChainSkills/miner1

computer$ ./startminer1.sh
...
 Open a second terminal and start the Geth console:
computer$ geth attach
...
>

Stop the mining to avoid generating too much blocks before the synchronisation process:

> miner.stop()
true

Retrieve the node information:

> admin.nodeInfo.enode

"enode://b8863bf7c8bb13c3afc459d5bf6e664ed4200f50b86aebf5c70d205d32dd77cf2a888b8adf4a8e55ab13e8ab5ad7ec93b7027e73ca70f87af5b425197712d272@[::]:30303?discport=0"

Step 3.2 Get Node info from miner #2

Repeat the process to retrieve the node information from the second miner.
Start the miner #2:
computer$ cd ~/ChainSkills/miner2

computer$ ./startminer2.sh
...
 Open a second terminal and start the Geth console:
computer$ geth attach ipc:./miner2/geth.ipc 
...
>

Stop the mining to avoid generating too much blocks before the synchronisation process:

> miner.stop()
true

Retrieve the node information:

> admin.nodeInfo.enode

"enode://41be9d79ebe23b59f21cbaf5b584bec5760d448ff6f57ca65ada89c36e7a05f20d9cfdd091b81464b9c2f0601555c29c3d2d88c9b8ab39b05c0e505dc297ebb7@[::]:30304?discport=0"

Step 4 – Pair the nodes

There are different ways to pair nodes.

Here, we illustrate how to define permanent static nodes stored in a file called “static-nodes.json“. This file will contain the node information of our miners.

Based on our environment, we will have the following content:

[
"enode://b8863bf7c8bb13c3afc459d5bf6e664ed4200f50b86aebf5c70d205d32dd77cf2a888b8adf4a8e55ab13e8ab5ad7ec93b7027e73ca70f87af5b425197712d272@192.168.1.39:30303",
"enode://41be9d79ebe23b59f21cbaf5b584bec5760d448ff6f57ca65ada89c36e7a05f20d9cfdd091b81464b9c2f0601555c29c3d2d88c9b8ab39b05c0e505dc297ebb7@192.168.1.39:30304"
]

You will notice that we have replaced the placeholder [::] with the IP address of our computer. The last part (?discport=0) has been removed.

Each node information is separated by a comma character.

Based on our example, the file “static-nodes.json” must be stored under the following location:

  • ~/ChainSkills/miner1
  • ~/ChainSkills/miner2

When the miner starts, the Geth process will process the file automatically.

Step 5 – Restart your miners

Stop and start the miners to ensure that they will properly reload the “static-nodes.json” file.

If you check the console of your miners, you should see a line mentioning the synchronisation process (“Block synchronisation started”):

...
I1219 01:12:03.223537 eth/downloader/downloader.go:326] Block synchronisation started
...

Step 6 – Check the synchronisation process

We check if the miners are properly paired.

Step 6.1 – Check from miner #1

Open the Geth console linked to the miner #1:

computer$ geth attach
...
>

Check which nodes are paired to miner #1:

> admin.peers
[{
 caps: ["eth/62", "eth/63"],
 id: "41be9d79ebe23b59f21cbaf5b584bec5760d448ff6f57ca65ada89c36e7a05f20d9cfdd091b81464b9c2f0601555c29c3d2d88c9b8ab39b05c0e505dc297ebb7",
 name: "Geth/miner2/v1.5.5-stable-ff07d548/darwin/go1.7.4",
 network: {
 localAddress: "192.168.1.39:59153",
remoteAddress: "192.168.1.39:30304"
 },
 protocols: {
 eth: {
 difficulty: 96831214,
 head: "0x0f8a3318a47429aee7a442cd1c3258eab7427b7fa1cb3c2a3e4bdf70ed6d8cf8",
 version: 63
 }
 }
}]

We can see that our node is paired to miner #2 identified by its IP address and its port number (30304).

Step 6.2 – Check from miner #2

Open the Geth console linked to the miner #2:

computer$ geth attach ipc:./miner2/geth.ipc 
...
>

Check which nodes are paired to miner #2:

> admin.peers
[{
 caps: ["eth/62", "eth/63"],
 id: "b8863bf7c8bb13c3afc459d5bf6e664ed4200f50b86aebf5c70d205d32dd77cf2a888b8adf4a8e55ab13e8ab5ad7ec93b7027e73ca70f87af5b425197712d272",
 name: "Geth/miner1/v1.5.5-stable-ff07d548/darwin/go1.7.4",
 network: {
 localAddress: "192.168.1.39:30304",
 remoteAddress: "192.168.1.39:59153"
 },
 protocols: {
 eth: {
 difficulty: 88045328,
 head: "0xa084d0c7f1a18120780c44bd4ba6c3ec237eb3feb0912bffd74ac1030246a723",
 version: 63
 }
 }
}]

We can see that our node is paired to miner #1 identified by its IP address and its port number (59153).

Step 7 -Validate the synchronisation

Let’s validate the synchronisation process by sending some ethers between accounts defined on each miner.

We are going to send 10 ethers between the following accounts:

  • miner #1(eth.coinbase) -> miner #2(eth.accounts[1])

eth.coinbase is the default account that receive the rewards for the mining process. In the miner #1, eth.coinbase is the same as eth.accounts[0].

First, make sure that the mining process is running on both miners.

Step 7.1 – Send ethers from Miner #1 to Miner #2

Start a Geth console linked to the miner #2, retrieve the address of the account[1] and check its initial balance:

computer$ geth attach ipc:./miner2/geth.ipc 
...
> eth.accounts[1]
"0xfa919b49ef34a821fb4cadfdfa5cc6593cb46fe1"

>  web3.fromWei(eth.getBalance(eth.accounts[1]))
0

The account #1 has no ether.

From the Geth console linked to the miner #1, send 10 ethers from the default account to the address of the account[1]:

computer$ geth attach
...
> eth.sendTransaction({from: eth.coinbase, to: "0xfa919b49ef34a821fb4cadfdfa5cc6593cb46fe1", value: web3.toWei(10, "ether")})

From the miner #1, check if the recipient has received the ethers:

> web3.fromWei( eth.getBalance("0xfa919b49ef34a821fb4cadfdfa5cc6593cb46fe1"))
10

From the miner #2, check that you have the same balance:

> web3.fromWei( eth.getBalance(eth.accounts[1]))
10

Step 7.2 – Send ethers from Miner #2 to Miner #1

We are going to send 2 ethers between the following accounts:

  • miner #2(eth.accounts[1]) -> miner #1(eth.accounts[1])

From the Geth console linked to the miner #1, check the initial balance of the account that will receive ethers:

computer$ geth attach 
...
> eth.accounts[1]
"0xae3ab39b3ebc425289dad620aece197a4a3f8940"

>  web3.fromWei(eth.getBalance(eth.accounts[1]))
0

The account #1 has 0 ether.

From the Geth console linked to the miner #2, send 2 ethers from the account #1:

computer$ geth attach ipc:./miner2/geth.ipc 
...
> eth.sendTransaction({from: eth.accounts[1], to: "0xae3ab39b3ebc425289dad620aece197a4a3f8940", value: web3.toWei(2, "ether")})

Error: account is locked
 at web3.js:3119:20
 at web3.js:6023:15
 at web3.js:4995:36
 at <anonymous>:1:1
Oops! We cannot send the transaction because the account #1 is locked.
To send transactions from an account, Ethereum requires that you unlock this account. The coinbase account is unlocked by our startminer.sh script thanks to the password file. But all other accounts are locked.
From the miner #2, unlock the account #1 with its password:
> personal.unlockAccount(eth.accounts[1], 'type your password')
true

From the miner #2, we are ready to send our transaction:

> eth.sendTransaction({from: eth.accounts[1], to: "0xae3ab39b3ebc425289dad620aece197a4a3f8940", value: web3.toWei(2, "ether")})
"0x02b1c360e3b094e8c7faac3d3f8def8eec6350a3d68a1b7e071827d1026221f0"

From the miner #2, check that our balance has changed (2 ethers plus some transaction fees):

> web3.fromWei( eth.getBalance(eth.accounts[1]))
7.99958

From the miner #1, check that the recipient has received the ethers:

> web3.fromWei( eth.getBalance(eth.accounts[1]))
2

Summary

Congratulations! You have synchronised your miners.

In part 5, we will synchronise the RPi node with these miners.

Set up the private chain – miners (3/6)

This tutorial is part of a series of articles introduced here.

Part 2 described how to install Ethereum on your computer.

Our development environment requires the setup of a private Ethereum chain.

NB: It is important that what we mean by “private Ethereum blockchain” in this context has absolutely nothing to do with the “private blockchain” championed by Hyperledger, Eris/Monax, or the recently announced Enterprise Ethereum Alliance. These are different codebases with different client applications, so they correspond to different implementations of a different set of concepts. They are private in the sense that they limit who has access to their chain. They are also called permissioned blockchains, and to be perfectly transparent, we wish they were not even called blockchains at all. In this series of articles, what we call “private blockchain” is a private instance of the Ethereum implementation. Same code base, same client application, different network identifier and genesis block. In other words, what we will come to create in this series is a chain with the same rules as the main chain, the same consensus algorithm too, but a different root block. This will become clearer in part 3. For more information about the 3-layer model and differences between concepts, implementations and instances, you can also watch our Devoxx talk and read this article.

Our private chain needs miners in order to validate and propagate blocks of transactions within the blockchain. Miners will also be used to generate ether to pay for the gas required to process transactions on the Ethereum blockchain. Note that this ether will only be usable within our private blockchain.

Unfortunately, the Raspberry pi is not powerful enough to be used as a miner. The mining process is just too intense. This is why we will deploy at least two miners on our computer.

Why two miners and not only one?

At the time of writing this tutorial, Geth has an issue when it is deployed in a private chain. If you have only one miner, some transactions will not be processed and will remain pending. Having two miners solves this issue (check here for more information about the issue). By the way, two miners make sense to apply the notion of consensus within the blockchain.

Let’s start by describing how to create the first miner.

Introduction

Before we describe the steps to start the miners, it is important to understand the requirements for each node to join the same private blockchain:

  • Each node will use a distinct data directory to store the database and the wallet.
  • Each node must initialize a blockchain based on the same genesis file.
  • Each node must join the same network id different from the one reserved by Ethereum (0 to 3 are already reserved).
  • The port numbers must be different if different nodes are installed on the same computer.

All the operations must be performed from your computer.

Step 1 – Create the datadir folder

When running a private blockchain, it is highly recommended to use a specific folder to store the data (database and wallet) of the private blockchain without impacting the folders used to store the data coming from the public blockchain.

From your computer, create the folder that will host your first miner:

computer$ mkdir -p ~/ChainSkills/miner1

Repeat the operation for the second miner:

computer$ mkdir -p ~/ChainSkills/miner2

Step 2 – Create the Genesis file

Each blockchain starts with a genesis block that is used to initialize the blockchain and defines the terms and conditions to join the network.

Our genesis block is called “genesis.json” and is stored under “~/ChainSkills” folder.

Create a text file under ~/ChainSkills, called genesis.json, with the following content:

{
 "nonce": "0x0000000000000042",
 "mixhash": "0x0000000000000000000000000000000000000000000000000000000000000000",
 "difficulty": "0x4000",
 "alloc": {}, 
 "coinbase": "0x0000000000000000000000000000000000000000",
 "timestamp": "0x00",
 "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
 "extraData": "ChainSkills Genesis Block",
 "gasLimit": "0xffffffff"
}

Among the parameters, we have the following ones:

  • difficulty: if the value is low, the transactions will be quickly processed within our private blockchain.
  • gasLimit: define the limit of Gas expenditure per block. The gasLimit is set to the maximum to avoid being limited to our tests.

Step 3 – Initialize the private blockchain

It’s time to initialize the private blockchain with the genesis block.
This operation will create the initial database stored under the data directory dedicated to each miner.

Step 3.1 – Initialize miner #1

Type the following command to create the blockchain for the first miner:
computer$ cd ~/ChainSkills
computer$ geth --datadir ~/ChainSkills/miner1 init genesis.json

I1226 00:44:16.572007 cmd/utils/flags.go:615] WARNING: No etherbase set and no accounts found as default
...
I1226 00:44:16.604262 cmd/geth/chaincmd.go:131] successfully wrote genesis block and/or chain rule set: 6e92f8b23bcdfdf34dc813cfaf1d84b71beac80530506b5d63a2df10fe23a660
The logs provide the following information:
  • you need a default account
  • the blockchain has been successfully created

If you list the content of the miner1 folder, you will notice the following subfolders:

  • geth: contains the database of your private blockchain (chaindata).
  • keystore: location of your wallet used to store the accounts that you will create on this node.

Step 3.2 – Initialize miner #2

Repeat the same operation to initialize the second miner by specifying its own target folder (~/ChainSkills/miner2):
computer$ cd ~/ChainSkills
computer$ geth --datadir ~/ChainSkills/miner2 init genesis.json

I1226 00:47:14.867704 cmd/utils/flags.go:615] WARNING: No etherbase set and no accounts found as default
...
I1226 00:47:14.890635 cmd/geth/chaincmd.go:131] successfully wrote genesis block and/or chain rule set: 6e92f8b23bcdfdf34dc813cfaf1d84b71beac80530506b5d63a2df10fe23a660

Step 4 – Create accounts

Let’s create some accounts for our miners.

Step 4.1 – Accounts for miner #1

Create the default account that will be used to run the node.

This account will receive all ethers created by the miner in the private blockchain. These ethers will serve to test our solutions by paying the gas required to process each transaction.

To create the default account for the miner #1, type the following command. Keep the password in a safe place:

computer$ geth --datadir ~/ChainSkills/miner1 account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase: 
Repeat passphrase: 
Address: {3e3753727dd6d965c0c696ea5619b8050ca89a49}
Add an additional account for testing purpose:
computer$ geth --datadir ~/ChainSkills/miner1 account new 
Your new account is locked with a password. Please give a password. Do not forget this password. 
Passphrase: 
Repeat passphrase: 
Address: {ae3ab39b3ebc425289dad620aece197a4a3f8940}
The wallet for these accounts is located right here:
computer$ ls -al ~/ChainSkills/miner1/keystore 
-rw------- 1 eloudsa staff 491 Dec 26 00:55 UTC--2016-12-25T23-55-58.526082355Z--3e3753727dd6d965c0c696ea5619b8050ca89a49
-rw------- 1 eloudsa staff 491 Dec 29 17:53 UTC--2016-12-29T16-53-48.822735465Z--ae3ab39b3ebc425289dad620aece197a4a3f8940
To list all accounts of your node, use the following command:
computer$ geth --datadir ~/ChainSkills/miner1 account list

Account #0: {3e3753727dd6d965c0c696ea5619b8050ca89a49} /Users/eloudsa/ChainSkills/miner1/keystore/UTC--2016-12-25T23-55-58.526082355Z--3e3753727dd6d965c0c696ea5619b8050ca89a49

Account #1: {ae3ab39b3ebc425289dad620aece197a4a3f8940} /Users/eloudsa/ChainSkills/miner1/keystore/UTC--2016-12-29T16-53-48.822735465Z--ae3ab39b3ebc425289dad620aece197a4a3f8940

Step 4.2 – Accounts for miner #2

Repeat the same operation to create the default account for the second miner. The difference lies in the target folder (~/ChainSkills/miner2).

The default account of miner #2:

computer$ geth --datadir ~/ChainSkills/miner2 account new 
Your new account is locked with a password. Please give a password. Do not forget this password. 
Passphrase: 
Repeat passphrase: 
Address: {22b2f81b46146aa3e1036fd3dc5cbcf12551b2db}

An additional account of miner #2:

computer$ geth --datadir ~/ChainSkills/miner2 account new 
Your new account is locked with a password. Please give a password. Do not forget this password. 
Passphrase: 
Repeat passphrase: 
Address: {fa919b49ef34a821fb4cadfdfa5cc6593cb46fe1}
To list all accounts of your node, use the following command:
computer$ geth --datadir ~/ChainSkills/miner2 account list

Account #0: {22b2f81b46146aa3e1036fd3dc5cbcf12551b2db} /Users/eloudsa/ChainSkills/miner2/keystore/UTC--2016-12-26T00-00-12.726226915Z--22b2f81b46146aa3e1036fd3dc5cbcf12551b2db

Account #1: {fa919b49ef34a821fb4cadfdfa5cc6593cb46fe1} /Users/eloudsa/ChainSkills/miner2/keystore/UTC--2016-12-29T16-59-28.053461367Z--fa919b49ef34a821fb4cadfdfa5cc6593cb46fe1

Step 5 – Prepare the miners

We are ready to start the miners from our computer and to mine some ethers that will reward our default accounts.

Step 5.1 – Miner #1: setup

Let’s start by creating a file that will contain the password for our default account, which is the first account we created for each miner. Create a password.sec file under ~/ChainSkills/miner1/ that contains the password you configured for the first account on miner1, in plain text.
To start the miner #1, we will require to run the following command:
computer$ geth --identity "miner1" --networkid 42 --datadir "~/ChainSkills/miner1" --nodiscover --mine --rpc --rpcport "8042" --port "30303" --unlock 0 --password ~/ChainSkills/miner1/password.sec --ipcpath "~/Library/Ethereum/geth.ipc"

The meaning of the main parameters is the following:

  • identity: name of our node
  • networkid: this network identifier is an arbitrary value that will be used to pair all nodes of the same network. This value must be different from 0 to 3 (already used by the live chains)
  • datadir: folder where our private blockchain stores its data
  • rpc and rpcport: enabling HTTP-RPC server and giving its listening port number
  • port: network listening port number, on which nodes connect to one another to spread new transactions and blocks
  • nodiscover: disable the discovery mechanism (we will pair our nodes later)
  • mine: mine ethers and transactions
  • unlock: id of the default account
  • password: path to the file containing the password of the default account
  • ipcpath: path where to store the filename for IPC socket/pipe
You can find more information about Geth parameters right here.
We recommend that you store the Geth command into a runnable script. In our example, this script is called “startminer1.sh” and is located here: ~/ChainSkills/miner1
#!/bin/bash

geth --identity "miner1" --networkid 42 --datadir "~/ChainSkills/miner1" --nodiscover --mine --rpc --rpcport "8042" --port "30303" --unlock 0 --password ~/ChainSkills/miner1/password.sec --ipcpath "~/Library/Ethereum/geth.ipc"
What’s the meaning of the parameter “ipcpath“?
If you want to manage your private blockchain via the Mist browser, you will need to create the “geth.ipc” file in the default Ethereum folder. Mist will detect that your private network is running and will connect to it rather than starting its own instance of Geth.

 Step 5.2 – Miner #1: start

Make the script runnable:

computer$ cd ~/ChainSkills/miner1 

computer$ chmod +x startminer1.sh

Let’s start the miner #1:

computer$ ./startminer1.sh
...
You will notice that the server and the mining process start.
You default account will receive ethers mined by the node.
If you start Mist, you will see that the default account is rewarded by ethers.
Remember that these ethers are fake money only available in your private blockchain and not on the live main chain. So, you are not (yet) rich 😉

Step 5.3 – Miner #1: JavaScript console

You can manage your miner using the Geth Javascript console.
For example, you can start and stop mining from the console or send transactions.
This console needs to be attached to a running instance of Geth.
Open a new terminal session and type “geth attach“.
If you want to start or stop the mining process, proceed as below:
computer$ geth attach
...
> miner.start()
...
> miner.stop()
...
You will notice that “geth attach” didn’t need any additional parameters. The reason is that the “geth.ipc” file is generated into the Ethereum default directory.
More commands for the JavaScript console are described right there.
To quit the attached console, simply use the exit command.

Step 5.4 – Miner #1: stop

To stop your miner, go to its console window and press CTRL-C.

Another option is to kill the Geth running process as illustrated here below:

computer$ ps aux | grep geth

eloudsa 43872 0.0 2.2 556717632 363492 s001 S+ 3:49PM 1:58.01 geth --identity miner1 --dev

computer$ kill -INT 43872

Thanks to the identity parameter, we are able to easily identify the miner instance we want to stop.

Now let’s continue with the second miner.

Step 5.5 – Miner #2: setup

To prepare the second miner, the instructions are similar to those described for the miner #1 with some minor changes.
First, create the “password.sec” file under ~/ChainSkills/miner2 that will contain the password of the default account of your miner #2:
Create the script “startminer2.sh“that will be stored at the following location: ~/ChainSkills/miner2
#!/bin/bash

geth --identity "miner2" --networkid 42 --datadir "~/ChainSkills/miner2" --nodiscover --mine --rpc --rpcport "8043" --port "30304" --unlock 0 --password ~/ChainSkills/miner2/password.sec
The difference from the first miner is about the following parameters:
  • identity: a specific name used to identify our miner
  • datadir: the folder related to the second miner
  • rpcport: a specific HTTP-RPC port number used to reach the miner #2
  • port: a specific network port number used to reach the miner #2
  • password: path of the file containing the password of the default account on the miner #2
You will notice that we have the same networkid (42). This will be useful later.
We are not providing the “ipcpath” parameter. This means that the “geth.ipc” will be created in the data directory of the miner.

Let’s start the miner #2:

computer$ cd ~/ChainSkills/miner2

computer$ chmod +x startminer2.sh

computer$ ./startminer2.sh
...

Step 5.6 – Miner #2: JavaScript console

As for miner #1, we can manage our node through the JavaScript console.
Unlike miner #1, we will have to attach the console to the running instance of Geth because its “geth.ipc” file has been generated in the “datadir” folder and not in the default Ethereum folder.
To attach the console, you can attach it by mentioning the path to the IPC-RPC interface (geth.ipc):
computer$ cd ~/ChainSkills

computer$ geth attach ipc:./miner2/geth.ipc
Another option is to attach the console to the HTTP-RPC interface:
computer$ geth attach http://127.0.0.1:8043
...

Be aware that these commands may have some limitations such as starting or stopping the mining process:

computer $ geth attach http://127.0.0.1:8043
Welcome to the Geth JavaScript console!

instance: Geth/miner2/v1.5.5-stable-ff07d548/darwin/go1.7.4
coinbase: 0x22b2f81b46146aa3e1036fd3dc5cbcf12551b2db
at block: 1294 (Sat, 31 Dec 2016 16:08:22 CET)
 modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0

> miner.stop()
ReferenceError: 'miner' is not defined
 at <anonymous>:1:1

First, you can see that the “miner” module is not specified in the list of modules: eth:1.0 net:1.0 rpc:1.0 web3:1.0

This is the reason why we have an error when we try to stop the mining process.

The following command is more useful:

computer $ geth attach ipc:./miner2/geth.ipc
Welcome to the Geth JavaScript console!

instance: Geth/miner2/v1.5.5-stable-ff07d548/darwin/go1.7.4
coinbase: 0x22b2f81b46146aa3e1036fd3dc5cbcf12551b2db
at block: 1417 (Sat, 31 Dec 2016 16:12:25 CET)
 datadir: /Users/eloudsa/ChainSkills/miner2
 modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 rpc:1.0 shh:1.0 txpool:1.0 web3:1.0

> miner.stop()
true

Here the miner can be managed through the console because the miner module is available from the IPC interface.

Keep this in mind if you have any trouble running a command from the Geth console.

Of course, you can define the modules available for the Geth console by mentioning them from the command line (ipcapi and rpcapi parameters).

For example, let’s change the Geth command by mentioning the “miner” module:

#!/bin/bash

geth --identity "miner2" --dev --networkid 42 --datadir "~/ChainSkills/miner2" --nodiscover --mine --rpc --rpcport "8043" --rpcapi "db,eth,net,web3,miner" --port "30304" --unlock 0 --password ~/ChainSkills/miner2/password.sec

Start the miner and open the console with the following command:

computer $ geth attach http://127.0.0.1:8043
Welcome to the Geth JavaScript console!

instance: Geth/miner2/v1.5.5-stable-ff07d548/darwin/go1.7.4
coinbase: 0x22b2f81b46146aa3e1036fd3dc5cbcf12551b2db
at block: 1473 (Sat, 31 Dec 2016 16:21:22 CET)
 modules: eth:1.0 miner:1.0 net:1.0 rpc:1.0 web3:1.0

Now the miner module is available from the Geth console!

You can manage the mining process:

> miner.stop()

true

This shows that by changing the Geth command, you can allow or restrict the usage of some APIs available from the HTTP-RPC or IPC-RPC interfaces.

Step 5.7 – Miner #2: stop

The principle is the same as described for the miner #1.

Step 6 – Send ethers within miner #1

To ensure that our mining node is properly installed, let’s try to send some ethers between accounts created on each miner.

Step 6.1 – Start the miner and its console

Let’s begin to start the miner #1:
computer$ cd ~/ChainSkills/miner1

computer$ ./startminer1.sh
...
 Open a second terminal and start the Geth console:
computer$ geth attach
...
>

Step 6.2 – Check balances

Get the default account with “eth.coinbase”:
> eth.coinbase
"0x3e3753727dd6d965c0c696ea5619b8050ca89a49"
List the accounts with “eth.accounts”:
> eth.accounts
["0x3e3753727dd6d965c0c696ea5619b8050ca89a49", "0xae3ab39b3ebc425289dad620aece197a4a3f8940"]
Of course, you will find addresses related to your environment.
Check balances with “eth.getBalance(<account id>)”:
> eth.getBalance(eth.coinbase)
2.48859375e+21

> eth.getBalance(eth.accounts[1])
0
The default account has plenty of ethers obtained during the mining process. We will use them to test our solutions.
Use the following command to get the balance in ether:
> web3.fromWei(eth.getBalance(eth.coinbase))
3008.125
By default, the balance is expressed in Wei that is the base unit of ether.
A converter is available here.

Step 6.3 – Stop mining

We stop the mining process to let us review the content of a transaction:

> miner.stop()
true

Step 6.4 – Send ethers

Through the Geth console, we send 10 ethers from account #0 to account #1:

> eth.sendTransaction({from: eth.accounts[0], to: eth.accounts[1], value: web3.toWei(10, "ether")})
"0x4ed5ef14cb5df5069059ac201a7294d810b89dcc05c0f60f970b87bff7faa5b6"
The command returns a transaction hash.
As the mining process has been stopped, we can review the list of pending transactions:
> eth.pendingTransactions
[{
 blockHash: null,
 blockNumber: null,
 from: "0x3e3753727dd6d965c0c696ea5619b8050ca89a49",
 gas: 90000,
 gasPrice: 20000000000,
 hash: "0x4ed5ef14cb5df5069059ac201a7294d810b89dcc05c0f60f970b87bff7faa5b6",
 input: "0x",
 nonce: 0,
 r: "0xdfd5465e315207e41d10a01189c41a1e9b42e7577cd256a100b3fa2157e9b0",
 s: "0x39b10c973c1e9764e9b3cfd98339164d65e78971c917ae24fe880a2a7c775cf6",
 to: "0xae3ab39b3ebc425289dad620aece197a4a3f8940",
 transactionIndex: null,
 v: "0x1c",
 value: 10000000000000000000
}]
We can easily identify the following information:
  • The sender address
  • The recipient address
  • The value to transfer (expressed in Wei)
  • The hash of the transaction
  • The cost of the transaction (gas x gasPrice) in Wei

Let the miner process the transaction:

> miner.start()
true

The transaction is processed:

> eth.pendingTransactions
[]

Check the balance of the recipient:

> web3.fromWei( eth.getBalance(eth.accounts[1]))

10

The recipient address has received 10 ethers.

Congratulations your miner # 1 is properly installed!
Let’s do the same for miner #2.

Step 7 – Send ethers within miner #2

Test your miner #2 by following the steps described for miner #1.
Keep in mind that the Geth console will have to be started using one of the following commands (change the port id according to your settings):
computer$ geth attach ipc:/Users/eloudsa/ChainSkills/miner2/geth.ipc

OR

computer$ geth attach http://127.0.0.1:8043

Step 8 – Send ethers between nodes of the private blockchain

Now, let’s try to send ethers between accounts of our miners.

We will send 5 ethers from the default account of the miner #1 to the account #1 of the miner #2.

Step 8.1 – Start miners

First, start both miners (miner #1 and miner #2).

Step 8.2 – Check balance from miner #2

From the Geth console attached to the miner 2, check the initial balance of the account 1:

> web3.fromWei(eth.getBalance(eth.accounts[1]))
10

Step 8.3 – Send Ethers from miner #1

From the console of the first miner, send 5 ethers from the default account (eth.coinbase) to the address of the account 1 in miner #2:

> eth.sendTransaction({from: eth.coinbase, to: '[REPLACE WITH ADDRESS OF ACCOUNT 1 IN MINER #2]', value: web3.toWei(5, "ether")})
"0x866db90bade948dbec0679b0a673807a74d6b3d94b3a7616dd86d72eb8a72e9b"

Step 8.4 – Check balance from miner #1

Check the balance from the Geth console attached to the miner #1:

> web3.fromWei( eth.getBalance("[REPLACE WITH ADDRESS OF ACCOUNT 1 IN MINER #2]"))
5

We have 5 ethers while we should expect 15 ethers.

Step 8.5 – Check balance from miner #2

Check the balance from the Geth console attached to the miner #2:

> web3.fromWei( eth.getBalance(eth.accounts[1]))
10

We have 10 ethers while we should expect 15 ethers.

Why?

Because our miners are not connected to the same blockchain.

Each miner is running its own version of the private blockchain. The transactions are not distributed within the same private blockchain.

To synchronise the blockchain, we have to pair the nodes with each other.

Summary

We have installed our miners but they are not synchronised with each other.

We will do that in Part 4 (what a teaser, man!).

 

Install an Ethereum node on a computer (2/6)

This tutorial is part of a series of articles introduced here.

In part 1, we have installed Ethereum on the RPi.

In this tutorial, we will describe the steps to install Ethereum on your computer.

Installing Geth on a computer is quite straightforward. The installation instructions are available for different target platforms, including Linux, Windows and Mac. You can find them right here.

Here below, we describe all the steps to set up Geth on a Mac, as it is the most common configuration amongst developers.

NB: It is important that what we mean by “private Ethereum blockchain” in this context has absolutely nothing to do with the “private blockchain” championed by Hyperledger, Eris/Monax, or the recently announced Enterprise Ethereum Alliance. These are different codebases with different client applications, so they correspond to different implementations of a different set of concepts. They are private in the sense that they limit who has access to their chain. They are also called permissioned blockchains, and to be perfectly transparent, we wish they were not even called blockchains at all. In this series of articles, what we call “private blockchain” is a private instance of the Ethereum implementation. Same code base, same client application, different network identifier and genesis block. In other words, what we will come to create in this series is a chain with the same rules as the main chain, the same consensus algorithm too, but a different root block. This will become clearer in part 3. For more information about the 3-layer model and differences between concepts, implementations and instances, you can also watch our Devoxx talk and read this article.

Step 1 – Homebrew

Homebrew is a package manager that you should install on your computer:

computer$ brew --version

If Homebrew is not installed on your computer, install it using the following command:

computer$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Step 2 – Install Geth

In this step, we install the Go implementation of Ethereum protocol (Geth).

Geth can be installed with the following commands:

computer$ brew update
computer$ brew upgrade

computer$ brew tap ethereum/ethereum
computer$ brew install ethereum

Check the version of Geth using:

computer$ geth version
Geth
Version: 1.5.7-stable
Git Commit: da2a22c384a9b621ec853fe4b1aa651d606cf42b
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.7.5
OS: darwin
GOPATH=
GOROOT=/usr/local/Cellar/go/1.7.5/libexec

Ensure that the version of Geth is the same as the one installed on your RPi. If not, upgrade your nodes.

Step 2.1 – Update Geth

If Geth was already installed on your computer, update it to the latest version:

computer$ brew update
computer$ brew upgrade

If you have trouble to update Geth, use the following commands to force the update:

computer$ brew tap ethereum/ethereum
computer$ brew unlink ethereum
computer$ brew install ethereum

Check the version of Geth:

computer$ geth version
Geth
Version: 1.5.7-stable
Git Commit: da2a22c384a9b621ec853fe4b1aa651d606cf42b
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.7.5
OS: darwin
GOPATH=
GOROOT=/usr/local/Cellar/go/1.7.5/libexec

Step 3 – Run Geth

Start Geth using the following command:

computer$ geth
I1217 19:38:35.080348 ethdb/database.go:83] Allotted 128MB cache and 1024 file handles to /Users/eloudsa/Library/Ethereum/chaindata
...
I1217 19:38:35.492125 p2p/server.go:342] Starting Server
...
I1217 19:38:47.719187 eth/downloader/downloader.go:326] Block synchronisation started
...

When you start Geth like that, without any command line argument, it immediately starts synchronizing with the main chain, which takes several days.

Press CTRL-C to stop the Ethereum server.

Summary

At this stage, Ethereum is installed on your computer and able to synchronise with the live chain (mainnet).

The blockchain (chaindata) is located right here:

  • Mac: ~/Library/Ethereum/chaindata
  • Windows: %APPDATA%\Ethereum
  • Linux: ~/.ethereum

Your accounts will be stored in a wallet located in this folder:

  • Mac: ~/Library/Ethereum/chaindata/keystore
  • Windows: %APPDATA%\Ethereum\keystore
  • Linux: ~/.ethereum/keystore

Congratulations! Your computer is an Ethereum node.

Part 3 describes how to setup a private chain and a miner node.

Create a private Ethereum blockchain with IoT devices (1/6)

Introduction

This series of tutorials will describe how to set up a private Ethereum blockchain that will be composed of a computer (miner) and one or several Raspberry PI 3 devices (nodes).

The objective is to build a development environment to learn about blockchain principles, to develop and test your own smart contracts before releasing them to the live chain (mainnet).

To make this project fun, we will integrate a RPi with a Smart Contract deployed on a private chain. The Smart Contract will be used to check if a user has enough tokens to use a service. The RPi will be used to visualize the status of the contract.

NB: It is important that what we mean by “private Ethereum blockchain” in this context has absolutely nothing to do with the “private blockchain” championed by Hyperledger, Eris/Monax, or the recently announced Enterprise Ethereum Alliance. These are different codebases with different client applications, so they correspond to different implementations of a different set of concepts. They are private in the sense that they limit who has access to their chain. They are also called permissioned blockchains, and to be perfectly transparent, we wish they were not even called blockchains at all. In this series of articles, what we call “private blockchain” is a private instance of the Ethereum implementation. Same code base, same client application, different network identifier and genesis block. In other words, what we will come to create in this series is a chain with the same rules as the main chain, the same consensus algorithm too, but a different root block. This will become clearer in part 3. For more information about the 3-layer model and differences between concepts, implementations and instances, you can also watch our Devoxx talk and read this article.

Conventions

We will use the following naming conventions to identify the devices on which the commands are entered:

computer$ A computer requires to perform initial configuration

pi$ Shell of the node running on the Raspberry PI device

Disclaimer

This series of tutorials doesn’t pretend to be comprehensive neither to give you a full introduction to Ethereum.

There are many ways to set up a private Ethereum blockchain from the most simplistic to the most complicated.

Here are some other examples:

  • Microsoft Azure and RPi  (here)
  • Docker image (here)
  • Pre-defined images such with EthRaspian (here)

Part 1: Install an Ethereum node on a Raspberry PI (RPi)

In this first part, we will describe the steps to transform your Raspberry PI 3 (RPi) into an Ethereum node.

The RPi is intended to act as a node connected to our private Ethereum blockchain. This means that it’s not required to install a full flesh operating system. A minimal image of Raspbian is good enough.

For information, there are pre-bundled RPi images that make it possible for you to transform your RPi into an Ethereum node.

One of these images is provided by the project EthRaspbian.

Here, we chose to build our Ethereum node without these prepared images. The objective is to show you all the required steps to do it from scratch.

Due to hardware limitations (CPU, memory), your RPi will not be able to mine ethers or to build new blocks of transactions.

Step 1 – Check your configuration

We will use a Raspberry PI 3 with a microSD of 16 Gb and a computer to prepare the microSD card.

The setup of the RPi will require the following hardware:

  • A Raspberry Pi
  • An SD Card with at least 16Gb
  • A LAN cable
  • A keyboard
  • A display

Step 2 – Format your SD Card

Insert your SD card on your computer and format it as FAT32.

On macOS, use the format ExFat in Disk Utility:

Step 3 – Write the Raspbian image

From your computer, download the Raspbian Jessie Lite image (minimal image) available here.

Unzip the file.

Write the image on the SD card by following instructions there. The process varies slightly depending on whether you are on a MacOS, Linux or Windows machine.

Here is an example for macOS (be very careful with the commands, every character matters):

computer$ diskutil list
...
/dev/disk5 (external, physical):
...

computer$ diskutil unmountDisk /dev/disk5

computer$ sudo dd bs=1m if=2016-11-25-raspbian-jessie-lite.img of=/dev/rdisk5

computer$ diskutil eject /dev/disk5

Ensure that your RPi is powered off.

Insert the SD card on you RPi and power on the device. Your RPi will boot the image.

Step 4 – Configure your RPi

Log in the RPi with the default credential is:

  • username: pi
  • password: raspberry

The RPi needs to be configured:

  • Change the keyboard layout (if required)
  • Set the timezone: required to allow a blockchain synchronisation between nodes
  • Enable SSH: securely access your RPi from your computer
  • Change your hostname: easily identified your RPi within your network (example: node1, node2, etc.)

We use raspi-config to setup all these settings:

pi$ sudo raspi-config

1) Timezone can be changed from the option "Internationalisation Options" and "Change Timezone".

2) Keyboard layout can be changed from the option "Internationalisation Options" and "Change Keyboard Layout". 

3) SSH can be enabled from the option "Advanced Options" then "SSH". 

4) The hostname can be changed from the option "Advanced Options" and then "Hostname. In our tutorial, we use the hostname "ethpinode1"

Press Finish and reboot your RPi:

p$> sudo reboot

Finally, you should really change the default password of the pi account:

pi$ passwd

Step 5 – Setup Wi-Fi

To setup your Wi-Fi interface, proceed as described below:

pi$ sudo nano /etc/wpa_supplicant/wpa_supplicant.conf
  1. Change the country code to set yours
  2. Add the following lines at the end of the file and replace the ssid with the name of your WiFi and psk with your WiFi’s password:
network={
 ssid="enter the SSID of your Wi-Fi network"
 psk="enter the password of your Wi-Fi network"
}

To apply your changes, restart your RPi or your Wi-Fi interface:

pi$ sudo ifdown wlan0
pi$ sudo ifup wlan0

You can now unplug  your LAN cable.

Step 6 – Connect to your RPI

To connect remotely to your RPi through SSH, you have to know the IP address of your RPi device.

You can find it by typing the following command from your RPi:

pi$ ifconfig

Or you can search it from your LAN by filtering IP adresses from a MAC  prefix address (b8:27:eb is the MAC prefix address of RPi devices):

computer$ arp -na | grep -i b8:27:eb
? (192.168.1.31) at b8:27:eb:1c:aa:94 on en0 ifscope [ethernet]


If you have several RPi, you can locate them using the “host” command:

computer$ host 192.168.1.31
31.1.168.192.in-addr.arpa domain name pointer ethpinode1.

At this stage, you can continue the setup of your RPI remotely from your computer using the “ssh” (“pi” is the username defined on your RPi):

computer$ ssh pi@192.168.1.31

It’s time to install Ethereum.

Step 7 – Install Geth

In this step, we are going to install the Go implementation of Ethereum client called Geth.

Connect to your RPi and retrieve the type of your CPU:

pi$ cat /proc/cpuinfo
...
model name : ARMv7 Processor rev 4 (v7l)
...

Go to the download page of Go-Ethereum by clicking here.

Select the tab Linux from the stable releases page.

Locate the row related to your CPU model and copy the link address . It’s recommended to retrieve the latest version of Geth.

In our example, we use Geth 1.5.7.

Back to your RPi, retrieve the bundle and extract the archive:

pi$ wget https://gethstore.blob.core.windows.net/builds/geth-linux-arm7-1.5.7-da2a22c3.tar.gz

pi$ tar zxvf geth-linux-arm7-1.5.7-da2a22c3.tar.gz 

Copy the Geth application to the /usr/local/bin folder:

pi$ cd geth-linux-arm7-1.5.7-da2a22c3

pi$ sudo cp geth /usr/local/bin

You can remove the archive and the uncompressed folder.

Check the version of Geth:

pi$ geth version
Geth
Version: 1.5.7-stable
Git Commit: da2a22c384a9b621ec853fe4b1aa651d606cf42b
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.7.4
OS: linux
GOPATH=
GOROOT=/usr/local/go

The same process can be applied to update an existing version of Geth.

Now, Ethereum is installed.

Let’s start it!

Step 8 – Run Geth

Start Geth using the following command:

pi$ geth
...
I1217 19:11:27.420830 p2p/server.go:342] Starting Server
...
I1217 19:11:39.816110 eth/downloader/downloader.go:326] Block synchronisation started

Press CTRL-C to stop the Ethereum server.

Summary

At this stage, Ethereum is installed on your RPi and able to synchronise with the live chain (mainnet).

The blockchain data (chaindata) is located right here: ~/.ethereum/chaindata

Your accounts will be stored in a wallet located in this folder: ~/.ethereum/keystore

Congratulations! Your RPi is an Ethereum node.

You can repeat these steps to setup additional RPi.

Part 2 describes how to set up Ethereum on a computer.

Shameless plug

While you are waiting for the next part of this series, we just wanted to let you know that we are currently preparing a full-blown online training about the development of distributed applications on Ethereum and we are looking for your feedback to figure out what you would like to see in this training. If you want to help us, you can take a few minutes to answer a survey here.

And if you just want us to keep you informed when the full online training program will be available, you can register to our mailing list on beta.chainskills.com.

We Need to Know… What You Need to Know!

As we announced during our Devoxx talk, Said and I are working on creating an online training about the blockchain as a software development platform, and more specifically about creating distributed applications (DApps) on top of Ethereum.

We decided to launch this initiative for several reasons.

First of all, because we are absolutely fascinated by this technology, but it is clearly in its infancy and it needs some more advanced tools, libraries and frameworks. And the only way it’s gonna have that is if more software developers get involved, figure out the gaps and contribute to expand the ecosystem.

Second of all because the field is wide open for killer applications and real projects that will demonstrate the incredible scope of this platform.

Last but not least, there is far too much blockchain discussion nowadays that is focused on private blockchains, because the banks are attracting all the attention with their misplaced money. And we really think it’s essential we stop wasting all this money in something that has way less potential.

On the other hand, most of the Ethereum and blockchain documentation out there is very fragmented or outdated. We couldn’t find any source of information that was at the same time:

  • comprehensive, to present all the information regarding what a blockchain is, how it works, and how you can develop on it, not just with a programming language but also with a full development environment
  • up-to-date, especially as all the tools and languages evolve very fast
  • progressive, taking you from a basic software development basis to fully functional distributed application development knowledge
  • and supported, meaning that you should get answers to all your questions as you are progressing though this complex learning experience.

And that’s exactly what we would like to create with this training program. And yes, we intend to make this a commercial training, because otherwise it would simply be impossible to answer all the questions from the students, and to update the material on a regular basis, while keeping the quality of it at a maximum.

Now the thing is if we really want to maximize the value of this program, we need to know where our future students are and what they would like to get from it. And that’s why we prepared a little survey for you.

Thanks a lot in advance for answering that survey and/or sharing it with all the developers you know who might be interested in our training program.

What Qualifies as a Blockchain?

Over the last couple of weeks, Said and I gave our first public talk about the blockchain as a development platform to a crowd of software developers gathered in Devoxx Morocco and Belgium. Our primary goal was to take a step back from the technical aspects, from how you write smart contracts, how you secure the data using cryptography, and make developers sensitive about what’s at stake, the use cases, the limitations and generally WHY you should be interested in the blockchain as a development platform. The video of our talk in Devoxx Antwerp is on Youtube:

One question that we were asked a lot had to do with all the different flavours of blockchains and which ones have the most potential. And it is a very important question indeed, because as software developers, we are used to learning new technologies, new languages, new frameworks, new libraries, new paradigms, but we know that those learning curves are big investments and betting on the right horse as soon as possible can be decisive in your career. And it’s even more critical for the blockchain because whatever the flavour, a blockchain is a network of value so the ones that will be used the most will concentrate most of the financial value as well.

The 3-layer Model

During this presentation, we introduced our 3-layer model to try and make sense of all the concepts surrounding the blockchain:

3-layer

Concepts

le-cordon-bleus-complete-cooking-techniques

At the most basic level of the blockchain, there is a set of concepts:

Where the web is a network of information, the blockchain is a network of value in the sense that it is designed to ease, accelerate and optimize the creation and transfer of assets between a number of individual or institutional participants connected to the network.

The network itself being more or less reliable depending on whether participants can join or leave the network at will, whether or not the links between participants are more or less resilient, whether or not participants themselves trust each other, the data representing the transfer of those assets should be distributed throughout the network instead of being stored in one central location.

In order for this data to be distributed, you cannot store just the latest snapshot but you are forced instead to record all the transformations applied to that data. This way, any participant that joins the network after a while can catch up with the rest of the network by downloading and applying all the transformations since the last one he had in his possession. That’s why all the blockchains are based on the concept of a distributed ledger from which the current state can always be deducted at any point in time.

Since any participant in the network can add his own transformations (also known as transactions) to the ledger, and the current state depends on which transactions are in the ledger but more importantly in what order, it is very important for all the participants to find an agreement on one common ordered list of transactions. The key mechanism by which participants find an agreement is called a consensus algorithm. In short, this algorithm defines the rules by which new transactions can be appended to the distributed ledger and recognized by all the participants as the one truth.

Since this consensus algorithm defines how the common truth is built, it is very important to determine who has control over this algorithm, in other words what is the governance model by which the ledger is determined. With a distributed data set like the one of the blockchain, the degree of decentralization of this consensus algorithm is absolutely paramount: a completely centralized consensus is very easy to corrupt even if it is easier to conceptualize. At the other end of the spectrum, a completely decentralized consensus algorithm is very hard to corrupt, but much more complex to implement and manage.

Another implication of the distribution is that you need a protocol to disseminate information in the network and for each participant to make sure that the information he receives from others is valid and genuine. And this is precisely where cryptography plays a key role in the blockchain.

Last but not least, those transactions listed on the network are essentially pieces of code that determine the conditions in which assets can be transferred from one participant to another, or more generically the conditions in which the state of the blockchain can change. These pieces of code are sometimes called smart contracts and they can be very straightforward and safe, or more sophisticated and risky.

Implementations

zest-screen-recipeThere are several possible implementations for each of those concepts, depending on the values and constraints of those writing the code that implements them. A blockchain implementation is mostly represented by a code base that determines how the generic concepts mentioned above become reality. For the most part, this code is the one of the software program that runs on each node participating in the network.

Bitcoin was the very first implementation of the concepts of the blockchain. It was created in a very specific context with a very practical goal in mind: to create a new monetary system as an alternative to the centralized banking infrastructure that produced the 2008 financial crisis. That’s why in the Bitcoin blockchain implementation, most nodes are full nodes that store the entire ledger (almost 100GB to this date), data is fully distributed, transparent and anonymous. The consensus algorithm is also fully decentralized in the sense that any node can decide to participate in the consensus or not. And it remains secure despite this openness by combining principles of cryptography, economics and game theory in a specific form of consensus algorithm called proof-of-work: each node participating in the consensus algorithm, also called miner, competes every 10 minutes  in a contest to produce the next block of transactions to be appended to the chain, this 10-minute delay being designed to minimize the interest of an attack. The first one to produce the next block gets rewarded with a freshly-created amount of bitcoins, the crypto-currency that is also the main transferred asset on the network. This way, since miners are interested in the asset growing in value, if they play against the network, they essentially cut the branch on which they are sitting. The Bitcoin blockchain implementation also has its own smart contract language called Script, but this language is voluntarily simplistic and lacks one key feature to make it Turing-complete (ie generic enough to implement any sort of algorithm): it doesn’t allow for loops. In a sense, Bitcoin is a specialized blockchain, designed primarily to solve a very specific problem. There are ways to somehow expand its functionalities beyond its original purpose of creating an alternate currency system, but they clearly come as an afterthought.

It is mainly because of those limitations in the smart contract language and in the speed of the network that another implementation called Ethereum was born. Its ledger is also fully distributed and its consensus algorithm completely decentralized in the form of a proof-of-work technique similar to the one of Bitcoin. But some key parameters are different in Ethereum. There is no pre-determined delay to produce blocks but on average they are produced every 17 seconds or so at the moment. In Bitcoin, only the miner that finds the next block in the main chain is rewarded. In Ethereum, you can also be rewarded a smaller amount of Ethers (the main crypto-currency on Ethereum) if you find a few blocks in a minority chain (uncles), which implies you have to wait for more blocks to be certain that a given transaction will stay in a specific order in the longer chain, but since those blocks are produced faster, the overall confirmation time is also shorter. In addition, the smart contract language in the Ethereum blockchain is Turing-complete, which means it is sophisticated enough to implement any kind of algorithm to decide how state can be transformed. Of course this sophistication comes at several costs. First of all it means that transactions can require a varying amount of storage space and processing power. In order to avoid the possibility for a fraudulent or badly designed contract to crush the network with infinite loops, every instruction incurs a small cost that has to be paid in transaction fees by the actor that triggers every transaction to the miner that mines the block containing the transaction. Second of all, the programming language used to write smart contracts has to be carefully manipulated in order to protect against nasty bugs that could create bugs that could jeopardize trust in the network and its currency. But with this sophisticated smart contract language, Ethereum is a much more generic blockchain, that looks a lot like a globally distributed computing surface.

When Bitcoin appeared, financial institutions, and especially banks were really scared of it. Then they started to understand that there could be opportunities for them to leverage some of its features to save time and money in their missions, especially where these missions involve collaborating with other institutions: compensation, international transfers, clearing, etc. The most interesting concept to them was the distributed ledger design. But the regulatory context in which they operate (Know-Your-Customer rules, anti-money laundering, etc.) made it hard to use a fully transparent and anonymous implementation. In addition, their position as financial intermediaries, their fiduciary duties and their business model made it very hard for them to give up on centralization of governance. That’s why most of them chose a more centralized variant of consensus algorithm with a permission model deciding which participants can join the network, and more importantly which participants are trusted with appending new blocks to the ledger. This trend gave birth to a series of implementations like Hyperledger, Corda/R3, Eris/Monax and many more. Since the consensus algorithm in these implementations is centralized, it relies on trusted parties that don’t need to be incentivized to participate in securing the network. So they don’t need to define a backing crypto-currency like Bitcoin or Ethereum. They can also offer a much higher processing speed for transactions.

In addition to those already well-known implementations, there are also plenty of implementations that are currently in development to try and tackle some of the problems and limitations identified with Bitcoin and Ethereum for example. It is a constantly evolving field and the landscape is still very much open.

Instances

strawberry-chocolate-cake-1For any given implementation, ie for any given code base, there can be many different instances. For example, you can take the code of the Bitcoin blockchain implementation, change a few parameters and initialize a brand new chain with its own backing crypto-currency. This is what we call an alternate coin, or altcoin.

Most of the existing altcoins are merely copying another implementation with very few customizations, and as such bring very little added value on top of their source implementation.

Others try to tackle some limitation of their source implementation by forking the code base and modifying some aspects of it, for example to enforce stronger anonymity as in the case of zCash.

There can also be forks that create another specialized blockchain to solve another specific problem than a monetary system. For example, namecoin was designed to create some sort of decentralized and distributed domain name service.

Also, when the code base of a given implementation changes slightly the rules by which new blocks can be appended to the chain, but not all the nodes participating in the network switch to the new version, we can observe a phenomenon called a chain fork: starting at a certain block, the chain starts splitting into 2 different versions of the ledger, effectively creating 2 variants of the initial crypto-currency incompatible with one another. That is for example what happened on the Ethereum blockchain after the DAO attack: a hard fork was implemented that split the chain into an ETH and an ETC chain. Most of the participants in the original Ethereum blockchain have switched to the new version of the algorithm and are mining ETH. But enough of the participants remained on the old version of the software, giving birth to the Ethereum Classic (ETC) crypto-currency and chain.

Last but not least, the exact same code base can be instantiated in several versions for testing purposes. For example, beside the main Ethereum chain, you also have a test Ethereum chain called Morden, on which crypto-currency is essentially monkey money, this chain being used by developers to test their smart contracts. And you can also create your own instance with a few colleagues or on your laptop for development purposes.

All those examples are here to illustrate that for each implementation, there can be many instances running either exactly the same code or a slightly changed code but on a different data set. And a given crypto-currency is only usable on a given instance.

By the way, on CoinMarketCap.com, you can follow more than 700 different crypto-currencies corresponding to as many blockchain instances.

So which implementations and instances qualify as real blockchains?

In other words, which implementations and instances have the most potential to still exist and concentrate a lot of value in the next 10 years?

For some purists, or Bitcoin maximalists are they are sometimes called, there is only Bitcoin. It is the purest, oldest and thus most mature implementation, and as such it deserves respect and you should think twice before questioning its power. And of course, resistance is all the more understandable as a lot of actors of Bitcoin have a lot of Bitcoins vested in the network, and when interests shifts to other implementations or instances, the currency potentially loses some of its value. For those in that community, there is no other viable consensus algorithm than proof of work, there is no other crypto-currency than Bitcoin, and loop-allowing smart contract languages are pure heresy because they completely endanger the safety of the assets exchanged on the network. To them, even the concept of the blockchain as a development platform makes very little sense, and they deploy treasures of creativity to extend the Bitcoin blockchain with mechanisms to implement some of the features it lacks (sidechains). And if there are limitations like the speed of the network, its scalability in terms of storage volume, etc. it can all be addressed with carefully thought out incremental improvements deployed in very slow forks agreed on after long debates in the community. To these people, even altcoins of Bitcoin, other instances of the same codebase are considered by default as potential scams or pump-and-dump schemes (or shitcoins as they call them).

Then there are those who are really amazed by the power of the concepts implemented by the Bitcoin blockchain, but would like to see them implemented in completely different ways. They still want to keep all of the design principles of the original, but they need more: more speed, more ease of use, more programmability, more extensibility. To them, Bitcoin was the first implementation, but there is no reason not to move forwards to tackle limitations of Bitcoin in a more radical way. Sure sophistication comes with its own risks and instability, but that’s merely the cost of progress. To those people, the blockchain really is a full-fledged distributed computing surface and development platform that opens the way to a whole new generation of software solutions and business models. The most popular blockchain implementation in that regard is Ethereum right now, but it doesn’t mean it is the final one. Ethereum has gone through a series of issues since May 2016 that have triggered heated debates between Bitcoin maximalists who basically said “I told you so”, Ethereum enthusiasts who said “Don’t worry, we’ll fix this, everything is under control”, and others who started to propose new chains to tackle Ethereum’s issues in more fundamental ways, like Synereo or Tezos.

line-in-the-sand

And then of course you have all these big companies, mostly financial institutions, who are working on what they call “private blockchains”. In our view, those private blockchains are a corporate attempt at shoving a new technology into old business models and cultures, ripping the stuff apart to make it fit instead of using the opportunity to completely rethink their business model based on what a full blockchain really enables. You guessed it, we are not really convinced by this approach. To us, it looks a lot like what Microsoft and AOL tried to do with the web at the very beginning, trying to apply a “walled garden” approach to it, creating their very own Internet of sorts. Or like what some companies are still trying to do with the cloud, hosting an internal cloud in their own datacenters, thus completely missing the point of mutualized resources and shared elasticity. As far as we are concerned, we don’t consider these initiatives as “real” blockchains. They are generally pushed by big consulting companies like IBM or R3 who saw an opportunity to sell consultants by the hour to implement very complex solutions to problems that could very well be solved by much simpler architectures. The problems these companies have are real, but they jumped at the hype too quickly, and we think they are going to end up realizing that:

  • Either they should completely rethink their business model and organization around an open blockchain implementation that leverages all its power in a distributed AND decentralized model
  • Or they can stay with their centralized culture and implement much simpler solutions to the problems they really have to remain competitive.

In conclusion, for us, Bitcoin is way too limited and complex to develop on top of to be considered as a long term solution for distributed application development. It is the most mature network right now, but we shouldn’t stop there. Open generic blockchains like Ethereum (and others) are way more promising to us on the long run, despite their issues and constant evolution. But private blockchains like the ones sported by big consulting companies and banks don’t present the most interesting path to us. We think that the blockchain is as much a political and cultural challenge as it is a technological one. It really gives us the opportunity to rethink business models and organization cultures and we want to explore that path. Time will tell if we bet on the right horse or not, and maybe we will be proven wrong, but for now, that’s our view on things.

A Few Resources to Get Started

Over the past few weeks, we have been reading and watching a lot of material about the blockchain and some of its variants (including the Bitcoin blockchain of course). The goal is not only to understand how it works technically, which will be necessary in order to understand how we can use it, but also to understand where it comes from, how it was created.

And we wanted to share with you some of the best resources we have found so far.

First things first. What are Bitcoin and the blockchain?  The first presentation is a great introduction that illustrates how Bitcoin and the blockchain are changing the rules of our world.

The second one is a presentation from Vinay Gupta, explaining in very clear terms how the blockchain came to be, from an historical standpoint, and how it fits in the timeline of data storage mechanisms. We love this talk because it doesn’t focus too much on the economic side of things but it sees the blockchain as a software paradigm, and a development platform, and that’s what we are really interested in as well, even though the currency part of it cannot be separated.

Vinay Gupta at Michel Bauwens & the Promise of the Blockchain from FIBER on Vimeo.

The third resource is also a video, a presentation by Andreas Antonopoulos, the author of Mastering Bitcoin, one of the THE reference books on how to develop on top of the Bitcoin blockchain apparently (we bought the book, didn’t read it yet). What we really love about this guy is that he is deeply invested in the Bitcoin blockchain, and yet he remains very curious and open-minded about all the evolutions of the blockchain technology. And as we are progressing in that space, we can see that a lot of people who have invested (or speculated) a lot on Bitcoin don’t really like the idea of being challenged by new tools and techniques about their core beliefs and assumptions. In that context, this talk and all of Andreas’ contributions are quite refreshing.

The last resource we wanted to mention in this blog post is a MOOC that I (Sebastien) have started following. It’s not free, it costs 830£ (by the way, given recent events, if you pay in euros or dollars, 830£ will be cheaper than usual), but so far, 2 weeks in, I must say that I’m pretty happy with the quality of the content.

I was kind of frustrated by the Ethereum tutorial, which seems pretty outdated and lacks a lot of details to get going. But the b9lab academy content is very practical and gives plenty of pointers to learn more around the main content. So far, it’s exactly what I was looking for. And one cool thing about this MOOC is that it will end with a certification that will be written to the Ethereum blockchain so that attendees can prove they followed through this course.

We will keep sharing some of the most interesting resources we find, both in English and French, about those topics, so if you have some resources to recommend yourself, feel free to leave a comment on this post.

Our Values

Said and I are starting a long and exciting journey with ChainSkills. And at the start of this kind of journey, it is often useful to take a step back to ask ourselves who we are, what we believe in, what our ambition is and what we want to achieve. This is what some call Mission/Vision/Values, but what we decide to call Identity/Ambitions/Project here, because we estimate that it makes more sense to think about it in that order, and there is too much ambiguity about these terms of mission, vision and values. The goal of this post is to lay the foundation of the identity of the team behind ChainSkills.

Pragmatism

We are both passionate about technology, but first as a means rather than an end in and of itself. For us, technology must always be useful and useable.

Useful because technology for itself, without concrete application, without an impact on life sounds like a huge waste given the infinite number of problems technology could help in solving.

Useable because the most elegant technology will always fail if it is not thought around its users, while keeping in mind their environments, their ecosystems and their mental models.

The thing is that when you speak about cryptography, security and networks, and mix in a little bit of economics and finance, as it is the case with crypto-currencies for example, it is very easy to get carried away by big abstract concepts, too much lingo and a certain fundamentalism that threaten this pragmatism.

We start from the assumption that, while making the best out of the benefits of a technology, it is sometimes necessary to find acceptable compromises to keep the technology useful and useable.

Innovation

At its core, technology is about making the impossible possible, and making the hard stuff easier. In doing so, it is not rare for it to upset business-as-usual, whether it is legal or fiscal rules, a business model or an industry. That’s what disruption is really all about.

When an ecosystem is disrupted, as it was the case recently with the taxi ecosystem and Uber for example, a naive reaction is to try and stop change in its course by blocking innovation or making it illegal.

And when politicians intervene, under the pressure of lobbying, they can overcompensate technological simplification with administrative complexity, such as what the Belgian administration is trying to do with models like Menu Next Door.

Our position regarding those trends is clear: we estimate that technology’s primary mission is to improve our lives, to make them easier and more rich, and we also estimate that nothing artificial should slow down or block that innovation. What we mean by artificial here is protecting abstract constructions such as employment or unfair competition. Innovation forces us to adapt, to evolve our beliefs and our vision of the world, and to grow as a civilization.

That’s why we will do everything in our power to minimize the compromises whose sole purpose would be to accommodate ancient administrative, legislative, competitive or fiscal models. The disruptive power of blockchain is too enormous to allow that.

Note that we are talking about minimizing compromises, and note about eliminating them, because we are aware that, at certain moments, such compromised will be inevitable to stay pragmatic.

And of course, what we don’t consider as artificial are the health, well-being and integrity of the recipients of this innovation, and as long and legal, fiscal and administrative rules will be designed primarily to protect those, we will abide by those rules unconditionally.

Lean

Said and I have both been immersed for years into the culture of Agile methodologies and software development. And today as entrepreneurs, we believe strongly in the values of Lean Startup.

Behind those lies the notion that we have access to limited and precious resources, whether they are time, energy, or more pragmatically financial means, and that it is our duty to use those resources with as much efficiency as possible, in order to go as far as possible.

In that regard, we have always leaned towards an experimental, incremental and iterative approach in the construction of our projects. We rarely try to do things perfectly on the first try, to think a problem through before moving forward. We prefer experimenting and observing empirically to hyper-theorizing. We value prototypes that do 80% of the job with 20% of effort. And we believe in the necessity to test and validate your hypotheses as soon as possible, before implementing them.

As in any new domain, our main goal here is not to get wealthy financially, but to learn, to understand and to integrate. Maybe one day, we will have learned enough to capitalize on this expertise and this understanding. But in a first approach, we want to explore and understand in the leanest and more pragmatic way possible.

Humanism

Blockchain technology was originally discovered in the context of a financial application with Bitcoin. It thus inherited some of its philosophical concepts that we don’t share entirely.

Most of today’s market finance is based on neo-liberal theories according to which human beings are cupid and selfish, and thus you have to embrace those trends fully and integrate them in the rules that regulate markets, betting on the fact that the sum of individual interests will always favor improvement in common interest.

We how two objections to those theories:

We have two major objections to these theories:

  • some major crises, including those that we are still going through today, tend to demonstrate that as stakes and inequalities get higher, individual interests can push people to do everything in their power to destroy interests of their peers, especially when it is easy to hide behind figures, regulations and anonymous structures. In such a situation where inequalities are too high, power tends to concentrate in the hands of a minority that threatens common interest in order to concentrate more and more power. We can only wonder about the solidity of this assumption that common interest is simply the sum of individual interests.
  • If it is true that survival instinct and adaption are parts of our genetic heritage and powerful forces, there is at least one other instinct that doesn’t seem used enough in big human interaction schemes like finance: gregarious instinct. If it is true that an individual will always do everything in his power to protect and improve his own life and the one of his lineage, he also understands the importance of the group and his own integration in this group to increase not only his own chances of survival but also his very quality of life. He instinctively knows that he needs others, and others need him, and with all the individual power in the world, he is nothing if he is isolated.

For these two reasons, we would like to witness the emergence of a system that pushes forward notions of reputation, accountability and mutual interest to go towards another macro-economic vision.

Besides, it is important to underline the fact that we do not adhere to this idea that there is a dichotomy between capitalism and socialism, and since socialism has been attempted and failed in the past, capitalism is the most advanced form of society there can be. We think there is another path, a path that would push forward some of the notions mentioned above but at a larger scale. And we think that blockchain technology, in its distributed and decentralized model, is an ideal platform to enable this kind of societal change.

Openness

The blockchain is a complex topic, that involves some advanced concepts in computer science, mathematics, economics, finance, etc. With such topics, it’s easy to get carried away in lingo and acronyms.

Moreover, with blockchain, it seems reasonable to observe that we are witnessing the dawn of a technological revolution, with its own “eldorado” effect, that gives birth to a whole generation of self-proclaimed and opportunistic experts who see this as a field to split, in which more people means smaller shares. Hence the temptation is big to exclude, to close and to place explorers of this new world in boxes: experts on one side, novices on the other.

As far as we are concerned, we don’t see this domain as a zero-sum game in which some have to lose for others to profit (more). To us, it looks like a gigantic field, whose potential is virtually unlimited, and that calls for an enormous diversity in vantage points and philosophical directions, given the large scope of potential changes.

That’s why we are convinced that yesterday’s novices are the experts of tomorrow, but more importantly that not everyone needs to be an expert in a field to have their say. We think everyone should be empowered to take ownership of these concepts to the best of their abilities, and our approach will always be to facilitate this access instead of restricting it.

Plus, in all humility, we ourselves are at the very beginning of our exploration of this vast domain, and we will thus go more naturally towards those who make information accessible and seek to include instead of excluding.

Our approach here will always be to discover together, to share what we think we understood, in good faith, with the community and to include a maximum of the constructive feedbacks we will receive to make them accessible in turn to the community.

Your comments, your corrections, your suggestions, your opinions and your questions will always be welcome on this platform.

Our goal with ChainSkills is not to create a secret society, an elite and closed group that would seek to protect its certainties and privileges.

We believe in the values of Open Source, sharing and open transparent collaboration. And this blog in the first manifestation of that spirit. By sharing our learning process, we hope it will help other explorers to move forward with these subjects, and that our shared experiences will produce a result far greater than their sum.

Of course, this will also apply to most of the products and experiments that will originate in these experiences, that will also be Open Source by default, except in special circumstances.

Lastly, this spirit of openness also applies to all those who will participate in that journey, without distinction of sex, age, beliefs or origin, provided they hold themselves to the same openness values.

No dogmatism

If blockchain was a religion, Satoshi Nakamoto would be its god, Vitalik Buterin its prophet. The Bitcoin whitepaper would be the Old Testament, the Ethereum whitepaper would be the New Testament. And there would be apostles and priests at war to spread the Word.

We don’t position ourselves in this logic at all. Our vision is that of a open-minded, pragmatic and non-dogmatic community, in which people don’t do things a certain way only because certain authorities said so. We are trying to understand, integrate and climb on the shoulders of our peers to build something greater and push forward practical applications that bring innovation in human societies.

In that regard, while completely respecting the expertise and the work of those who came before us, we naturally mistrust the argument of authority, behind which we will always try to understand the motivations behind the choices, even if it means exploring other choices and thinking out of the box.

Last but not least, we don’t see ourselves as apostles of the blockchain, we don’t position ourselves as owners of a knowledge to pass on, but more as explorers with and for the community.


Of course, if you have questions, remarks, corrections or suggestions, comments are open, and so long as they stay constructive, they will be more than welcome.


Said & Sébastien

Hello World!

We are Sebastien Arbogast and Said Eloudrhiri, we are software engineers and we welcome you to our new project.

ChainSkills was born out of a growing interest in a new concept called the blockchain. To put it simply, the blockchain is a relatively new technology that makes it possible to collaborate on a data set in a completely decentralized and distributed way.

Before the blockchain, the best way to have several actors collaborate on a common data set was to store that data set on a central server, and to have each actor connect to that central database. Then all you had to do was to secure access to this database in such a way that only certain actors could read or modify the data (authentication) and you could determine which actor could access which data in which way (authorization). Also, you had to put some code in front of the database in order to orchestrate access to that database. Security and integrity of the database was thus enforced by centralization. But this approach poses a certain number of problems:

  • If some malevolent actor wants to corrupt that data, all he has to do is to gain access to this central database and all other actors will access corrupted data (integrity)
  • If some authority wants to censor that data, they can simply change the rules to access the central database, or even alter the central database
  • If the central database is destroyed because of a technical failure of some sort, then all the actors lose access to that database and the system can suffer major damage, if not complete destruction

In addition, in principle, this way of organizing software systems is in direct contradiction with the design principles of the Internet it is using as an infrastructure. And it just so happens that the Internet was designed in a completely decentralized and distributed fashion precisely to avoid those problems, at a network level.

The blockchain is essentially an attempt at producing a software database paradigm that also works in a completely decentralized and distributed fashion, all while keeping the ability to maintain integrity, security and accessibility on that shared data.

Historically, the very first implementation of that new paradigm was part of the Bitcoin digital currency created in 2009. But since then, a lot of people have started to realize the opportunity to use this same technique as a foundation for many other use cases.

And this is where we come from.

We are not cryptography experts.
We are not cypherpunks.
We are not crypto-currency traders.
We are not security researchers.
We are not dogmatic, we don’t do things a certain way only because Satoshi Nakamoto or Vitalik Buterin or any other authority said so.

We are software developers.
We are curious experimenters.
We are lean entrepreneurs.
We are machine creators.
We are French-speaking (and a little bit of English too, obviously)

What we are going to do in this blog is simple: we want to get to understand more about this new paradigm created by the concept of a blockchain. We want to learn more about it, about all its subtleties, about the opportunities it creates, about its limitations too. And of course we want to explore these opportunities in a pragmatic and experimental way.

And we will use this blog to report our findings, share our experiments, document our progress and get the community’s feedback.

Our approach here is that of an open learning experience.

Of course, we already have plenty of concrete ideas about how we could use that new knowledge some day. But we also approach this with a humble perspective: we have to acquire the knowledge first, to know whether our ideas make sense. And if they do, we will share those ideas with you too.

So buckle up, and enjoy the ride.