Introduction
In the first post, I explained there were four aspects of a blockchain that this series would explore with a backing implementation provided by the Ardan blockchain project.
The first post focused on how the Ardan blockchain provides support for digital accounts, signatures, and verification. In this second post, I will focus on transaction distribution and synchronization between different computers (blockchain nodes).
Source Code
The source code for the Ardan blockchain project can be found at the link below.
https://github.com/ardanlabs/blockchain
It’s important to understand this code is still a work-in-progress and that things are being refactored as more is learned.
High-Level Overview
Here is a quick high-level overview of the workflow I will be describing in this post.
Figure 1: High-Level View
Figure 1 shows how transactions flow through the Ardan blockchain. At a high-level, a transaction is submitted by the Ardan wallet to a blockchain node. The node accepts the transaction and will store it in an in-memory cache called the mempool. Then the transaction is signaled to a goroutine so it can be sent to other blockchain nodes in the network, where the transaction is stored in their mempool.
There are lots of little details to make all this work and I will describe this flow by walking through the code.
Submitting a Transaction
The Ardan blockchain implements a HTTP based REST API for communication with wallets and between blockchain nodes. This is done to keep things simple since this is just a reference implementation. As mentioned in the previous post, Ethereum uses a JSON-RPC protocol for this same communication.
Listing 1: SubmitWalletTransaction - Handler
01 app.Handle(http.MethodPost, "v1/tx/submit", pbl.SubmitWalletTransaction)
02
03 func (h Handlers) SubmitWalletTransaction(
04 ctx context.Context, w http.ResponseWriter, r *http.Request) error {
05
06 var signedTx storage.SignedTx
07 if err := web.Decode(r, &signedTx); err != nil {
08 return fmt.Errorf("unable to decode payload: %w", err)
09 }
10
11 if err := h.State.SubmitWalletTransaction(signedTx); err != nil {
12 return v1.NewRequestError(err, http.StatusBadRequest)
13 }
14
15 resp := struct {
16 Status string `json:"status"`
17 }{
18 Status: "transactions added to mempool",
19 }
20
21 return web.Respond(ctx, w, resp, http.StatusOK)
22 }
Listing 1 shows the HTTP handler function that receives a new signed transaction from the Ardan wallet. On line 01, you see how the handler function is bound into a route of v1/tx/submit using the method POST. Then on lines 03 and 04, you see the declaration of the handler function. We used the Ardan web framework from Ultimate Service.
On line 07, the JSON document provided in the POST call is decoded into a value of type SignedTx. Then with the signed transaction in hand, the SubmitWalletTransaction method from the blockchain/state package is called on line 11. If that call succeeds, the wallet receives a 200 and a simple message that the transaction has been added to the mempool.
Listing 2: SubmitWalletTransaction - blockchain/state
01 func (s *State) SubmitWalletTransaction(signedTx SignedTx) error {
02 if err := s.validateTransaction(signedTx); err != nil {
03 return err
04 }
05
06 tx := storage.NewBlockTx(signedTx, s.genesis.GasPrice)
07
08 n, err := s.mempool.Upsert(tx)
09 if err != nil {
10 return err
11 }
12
13 s.worker.signalShareTransactions(tx)
. . .
19 return nil
20 }
Listing 2 shows the SubmitWalletTransaction method from the blockchain/state package that was called by the handler in listing 1. On line 02, the signed transaction is validated against several checks. The main check is to verify the transaction came from a wallet that supports the Ardan blockchain and that the signature is valid for the transaction data that was submitted. Other checks are also performed to validate field level data is formatted correctly.
On line 06, a block transaction is constructed from the signed transaction and the gas fee defined in the genesis file. The block transaction is used for appending fees and data that need to be associated with the signed transaction when stored inside a block. Then on line 08, the block transaction is added to the mempool and on line 13, a worker goroutine is signaled to share the block transaction with other known blockchain nodes.
The mempool is an in-memory cache of submitted transactions waiting to be selected for the next block. With the Ardan blockchain, nothing happens until there are enough transactions in the mempool to start a mining operation. Empty blocks are never mined, however this does happen on other blockchains like Ethereum and Bitcoin.
To better understand the nature and structure of the mempool, a basic understanding of mining is important.
Mining
Mining in the Ardan blockchain is a competition between blockchain nodes based on [proof of work] (https://en.wikipedia.org/wiki/Proof_of_work). At roughly the same time, all nodes pick a set of transactions from the mempool, group them together into a data structure called a block, and attempt to find a proper hash for that block. In the case of the Ardan blockchain, a proper hash has six leading zeros. Finding this proper hash requires changing the block’s nonce field and producing a new hash (over and over again) until a nonce is found that produces the proper hash.
Listing 3: Example Hash and Nonce
"Hash" : "00000064b48f2cf91a97f07681a3959227c3d73ea00a3e74df7b229443dcdda0"
"Nonce": 5887799674285008887
"Hash" : "0000006a54a3a0c07108aa5a919d13d657b70c595de8e2e768858c14718bae58"
"Nonce": 6676248792032277910
Listing 3 shows examples of what the Ardan blockchain defines as a proper hash. I took these hashes from the blockchain blocks file in the project. You can also see the nonce that helped produce the hash for these blocks. The strategy of selecting a different nonce on each attempt is part of the process. The first node that finds a nonce that produces a proper hash receives a reward, the rest of the nodes have to eat the cost of competition.
Note: A nonce is a value that is only ever used once for a particular thing. Blocks and Transactions have a nonce value and they are defined as an unsigned integer. Watch this video to learn more.
Once a node is successful in finding a nonce that produces a proper hash, they write that block (with the nonce) into their local chain, remove the transactions grouped inside this block from their mempool, and share the block with other nodes . When the other nodes receive this new block, they stop mining, validate the block, include the block in their chain, and also remove the transactions grouped inside this block from their mempool.
Note: On Ethereum mining happens roughly every 12 to 14 seconds.
This proof of work based process is what drives the consensus mechanics for the Ardan blockchain.
- Each node performs heavy computational work to find a proper hash.
- When a node finds a solution, the block is shared with the rest of the nodes.
- The nodes validate the block and accept it into their chain.
- If the majority of the nodes accept the block, you have consensus.
The proof of work here is finding the right nonce to find a proper hash. This takes time and energy, and it can’t happen by accident.
Mempool Package
It’s not until a transaction is mined into a block, and that block is accepted by the majority of other blockchain nodes on the network, does it constitute a committed transaction. While the transaction is waiting in the mempool, it’s just pending.
Wallets submit transactions to a single blockchain node of their choosing and those transactions have to be stored and shared across all nodes participating in the network. In the Ardan blockchain, a package named mempool was written to support this functionality.
Listing 4: Mempool Type
01 package mempool
02
03 type Mempool struct {
04 pool map[string]storage.BlockTx
05 mu sync.RWMutex
06 selectFn selector.Func
07 }
Listing 4 shows the data structure used by the Ardan blockchain for the mempool. The data structure consists of a map named pool that stores the set of submitted transactions. It also contains a read-write mutex to provide safe concurrent access to the map, and a function that implements a select algorithm for picking transactions for the next block.
Listing 5: Selector Function
01 package selector
02
03 type Func func(
04 transactions map[storage.Account][]storage.BlockTx,
05 howMany int,
06 ) []storage.BlockTx
Listing 5 shows the declaration of the Func type from the selector package. You can see the function accepts a set of block transactions organized by account and the number of transactions to return. The default selector function organizes the transactions by nonce and tip per account. The implementation of the default selector function is beyond the scope of this post, but I will talk about it in the future.
The API for constructing a mempool, upsert, and delete is fairly mundane and provides the basic facilities you would imagine.
Listing 6: Construct
01 func New() (*Mempool, error) {
02 return NewWithStrategy(selector.StrategyTip)
03 }
04
05 func NewWithStrategy(strategy string) (*Mempool, error) {
06 selectFn, err := selector.Retrieve(strategy)
07 if err != nil {
08 return nil, err
09 }
10
11 mp := Mempool{
12 pool: make(map[string]storage.BlockTx),
13 selectFn: selectFn,
14 }
15
16 return &mp, nil
17 }
Listing 6 shows how to construct a mempool for use. Notice the use of pointer semantics for the return type. This is important since the struct type has a mutex. If you make a copy of a mutex, you have a different mutex.
The New function on line 01 defaults to using the tip strategy by passing selector.StrategyTip to the NewWithStrategy function on line 02. The NewWithStrategy function retrieves the tip selector function on line 06, and then on line 11, a new mempool is constructed.
Listing 7: Upsert
01 func (mp *Mempool) Upsert(tx storage.BlockTx) (int, error) {
02 mp.mu.Lock()
03 defer mp.mu.Unlock()
04
05 key, err := mapKey(tx)
06 if err != nil {
07 return 0, err
08 }
09
10 mp.pool[key] = tx
11
12 return len(mp.pool), nil
13 }
Listing 7 shows the Upsert method. This method is using a mutex write lock before allowing any changes to the map. Then on line 05, the mapKey function is used to construct an unique map key for the specified transaction. With the key, the transaction can be stored or replaced.
Listing 8: mapKey
01 func mapKey(tx storage.BlockTx) (string, error) {
02 account, err := tx.FromAccount()
03 if err != nil {
04 return "", err
05 }
06
07 return fmt.Sprintf("%s:%d", account, tx.Nonce), nil
08 }
Listing 8 shows the mapKey function. On line 02, the account that signed the transaction is extracted. Then on line 07, the account is combined with the nonce of the transaction to construct a unique key.
The nonce starts out at zero for new accounts and then for each transaction submitted by that account, the account’s wallet will increment the nonce by one. The nonce is very important because it makes sure multiple transactions by the same account are executed in the proper order. The same nonce per account can’t be used twice and it always needs to be larger than the nonce from the last mined transaction. The blockchain node will validate the nonce for every new transaction submitted by an account. The nonce also allows an account to replace a transaction that is “stuck” in the mempool.
The idea of a transaction being “stuck” in the mempool is a real problem at times. If blockchain nodes don’t want to select your transaction for any reason (such as the tip being too small compared to other transactions), a transaction can stay in the mempool for a long time. To fix this problem, you can send a new transaction with the same nonce as the stuck transaction. This would allow you to replace the existing transaction in the mempool, giving you the ability to increase the size of the tip.
Listing 9: Delete
01 func (mp *Mempool) Delete(tx storage.BlockTx) error {
02 mp.mu.Lock()
03 defer mp.mu.Unlock()
04
05 key, err := mapKey(tx)
06 if err != nil {
07 return err
08 }
09
10 delete(mp.pool, key)
11
12 return nil
13 }
Listing 9 shows the Delete method. This method is similar to Upsert in terms of generating the map key to perform the delete operation. With the key, a map delete operation can take place.
Listing 10: PickBest
01 func (mp *Mempool) PickBest(howMany int) []storage.BlockTx {
02 m := make(map[storage.Account][]storage.BlockTx)
03 mp.mu.RLock()
04 {
05 if howMany == -1 {
06 howMany = len(mp.pool)
07 }
08
09 for key, tx := range mp.pool {
10 account := accountFromMapKey(key)
11 m[account] = append(m[account], tx)
12 }
13 }
14 mp.mu.RUnlock()
15
16 return mp.selectFn(m, howMany)
17 }
18
19 func accountFromMapKey(key string) storage.Account {
20 return storage.Account(strings.Split(key, ":")[0])
21 }
Listing 10 shows the PickBest method. This is the function that selects transactions from the mempool that will be used for mining the next block. Each blockchain can choose transactions in many different ways. In Ethereum, the only rule is that address nonce ordering is respected. Geth, Ethereum’s most popular client, uses gas price and time for its default ordering method. But that’s simply the default. Transaction ordering can be further optimized.
This method first copies all the transactions for each account into separate slices. Then the method relies on the selector package for the different transaction selection algorithms. On line 16, you can see the call to the configured selection function. Making this configurable allows you and others to experiment with different selection strategies.
Sharing Transactions
It’s critical that the mempool of each blockchain node is synced and acts like a centralized cache of pending transactions. It allows for more blockchain nodes to select the same transaction for the next block, which in turn provides more opportunity for the transaction to be mined.
Look at the method used to accept a new transaction from the Ardan wallet again.
Listing 11: SubmitWalletTransaction - blockchain/state
01 func (s *State) SubmitWalletTransaction(signedTx SignedTx) error {
02 if err := s.validateTransaction(signedTx); err != nil {
03 return err
04 }
05
06 tx := storage.NewBlockTx(signedTx, s.genesis.GasPrice)
07
08 n, err := s.mempool.Upsert(tx)
09 if err != nil {
10 return err
11 }
12
13 s.worker.signalShareTransactions(tx)
. . .
19 return nil
20 }
Listing 11 shows the SubmitWalletTransaction method from the blockchain/state package. Look at line 13, this is where a worker goroutine is signaled to share the received transaction.
Listing 12: signalShareTransactions - blockchain/state/worker
01 func (w *worker) signalShareTransactions(blockTx storage.BlockTx) {
02 select {
03 case w.txSharing <- blockTx:
04 w.evHandler("worker: share Tx signaled")
05 default:
06 w.evHandler("worker: queue full, transactions won't be shared.")
07 }
08 }
Listing 12 shows the signalShareTransaction method of the worker type which is called in listing 10. This code attempts to signal the block transaction through the txSharing channel. If that operation is going to cause the sending goroutine to block, the default option is there to prevent the blocking. To minimize dropping signals, the txSharing channel is constructed as a buffered channel.
Listing 13: runWorker - blockchain/state/worker
01 const maxTxShareRequests = 100
02
03 func runWorker(state *State, evHandler EventHandler) {
04 state.worker = &worker{
05 txSharing: make(chan storage.BlockTx, maxTxShareRequests),
. . .
10 }
Listing 13 shows the runWorker function that is used to initialize the worker goroutines that perform the different concurrent operations required to run the node. On line 05, you can see the construction of the txSharing channel. It’s a buffered channel of 100, meaning 100 transactions can be pending to be sent to other nodes before the signalShareTransaction method will begin to drop transactions.
Note: The number 100 is currently an arbitrary number and more engineering needs to be done with this signal. It works for now because this reference implementation doesn’t have any real production level traffic. It would be nice to batch these transactions and share more than one transaction across the network at a time, maybe on a well defined interval. It’s important there is very little latency when sending a new transaction to other nodes.
Listing 14: shareTxOperations - blockchain/state/worker
01 func (w *worker) shareTxOperations() {
02 for {
03 select {
04 case tx := <-w.txSharing:
05 if !w.isShutdown() {
06 w.runShareTxOperation(tx)
07 }
08 case <-w.shut:
09 w.evHandler("worker: shareTxOperations: received shut signal")
10 return
11 }
12 }
13 }
Listing 14 shows the shareTxOperations method that runs as a separate goroutine to receive signals for the txSharing channel and perform the sharing operation. The entire function is in an endless loop listening on two channels, one on line 04 to receive a block transaction to send to other nodes, and one on line 08 for terminating the goroutine.
When a signal is received on the txSharing channel, the runShareTxOperation method is executed to perform the HTTP calls.
Listing 15: shareTxOperations - blockchain/state/worker
01 func (w *worker) runShareTxOperation(tx storage.BlockTx) {
02 for _, peer := range w.state.RetrieveKnownPeers() {
03 host := fmt.Sprintf(w.baseURL, peer.Host)
04 url := fmt.Sprintf("%s/tx/submit", host)
05
06 if err := send(http.MethodPost, url, tx, nil); err != nil {
07 w.evHandler("worker: runShareTxOperation: WARNING: %s", err)
08 }
09 }
10 }
Listing 15 shows the runShareTxOperation method. On line 02, a loop is declared to iterate over all the known blockchain nodes (or peers). Then on line 04, the url needed to talk to the individual peer is constructed. Finally on line 06, the block transaction is sent to the peer.
The list of known peers is constantly updated by a different goroutine on a timer. This allows each blockchain node to eventually have a complete list of nodes over time.
Each peer has a route and a handler defined to receive the block transaction.
Listing 16: SubmitNodeTransaction - Handler
01 app.Handle(http.MethodPost, "v1/node/tx/submit", prv.SubmitNodeTransaction)
02
03 func (h Handlers) SubmitNodeTransaction(
04 ctx context.Context, w http.ResponseWriter, r *http.Request) error {
05
06 var tx storage.BlockTx
07 if err := web.Decode(r, &tx); err != nil {
08 return fmt.Errorf("unable to decode payload: %w", err)
09 }
10
11 if err := h.State.SubmitNodeTransaction(tx); err != nil {
12 return v1.NewRequestError(err, http.StatusBadRequest)
13 }
14
15 resp := struct {
16 Status string `json:"status"`
17 }{
18 Status: "added",
19 }
20
21 return web.Respond(ctx, w, resp, http.StatusOK)
22 }
Listing 16 shows the route and handler for accepting a new transaction from a blockchain node. On line 06, the post data is decoded into a block transaction and then on line 11, the SubmitNodeTransaction method is called to add the new block transaction to the mempool.
Listing 17: Submit Node Transaction - blockchain/state
01 func (s *State) SubmitNodeTransaction(tx storage.BlockTx) error {
02 if err := s.validateTransaction(tx.SignedTx); err != nil {
03 return err
04 }
05
06 n, err := s.mempool.Upsert(tx)
07 if err != nil {
08 return err
09 }
. . .
14 return nil
15 }
Listing 17 shows the SubmitNodeTransaction method from the blockchain/state package that was called by the handler in listing 16. On line 02, the transaction is validated and then on line 06, the block transaction is added to the mempool. The handler in listing 16 and this method provide the support to keep the mempool in sync across blockchain nodes in the network.
Conclusion
This second post focused on how the Ardan blockchain provides support for transaction distribution and synchronization between different computers (blockchain nodes). This represents the second of the four aspects of a blockchain that this series will explore with a backing implementation provided by the Ardan blockchain.
In the next post, I will share how the Ardan blockchain provides redundant storage of a single ledger on different computers.