The Hedera Consensus Service (HCS)

I started this leg of my journey by going over the Hedera Consensus Service white paper. After going through it a couple of times, I decided I needed more information. Though informative, the information is a bit too brief for my purposes. I’m looking at this from a developer’s utility viewpoint, and while there is interesting content in the white paper, and the basic facts about the HCS are present, there’s not enough solid “here’s how” or even “here’s why” content to get me going.

I ended up finding a better resource. Before I write about that, though, I think it prudent to go over briefly what I found in the white paper.

What is the HCS?

The HCS provides the timestamping and ordering of all events on Hedera. Using the hashgraph consensus algorithm, it guarantees fast, inexpensive, secure and disinterested consensus. There is (at this writing) a main network of 27 decentralized nodes that actively participate in the consensus (the mainnet). Augmenting mainnet is a network of many mirror nodes which can report on transactions only -- they don’t participate in consensus operations. More about mirror nodes in a bit.

Through the consensus algorithm, the mainnet nodes exchange “gossip” whenever transactions occur. They also gossip about the gossip they have been doing. This concept is documented pretty well on the Hedera web site. Gossip is the key feature which enables immediate finality and fair ordering of transactions. The main things for me to remember is that when any transaction is received by any mainnet node, it is final, and further, all transactions are sequenced in the correct order.

And I don’t need to take this on faith. I can appeal to a random set of any mainnet and mirror nodes to check the transaction.

Messages, Transactions, Topics, and Queries

As far as I can gather from the docs, message and transaction are effectively synonyms. For my purposes, I’m going to use transaction as that message sent to a mainnet node that’s intended to do something, query as that message sent to a mainnet or mirrornet node that’s intended to learn something, and message whenever I feel that it doesn’t matter.

HCS lives and breathes via messages. Transactions are issued specifying a single topic. Topics are globally unique tags, public or private, that can represent any concept. Rugby. Cooking. FY23 Earnings. Yamaha fenders. Whatever. Queries can specify topics. Mirrornet nodes can be built to listen to only certain topics.

Mainnet vs Mirror Node Networks

There are two major segments to the Hedera network:

Mainnet, though which transactions are generated, ordered and finalized. The mainnet has a limited number of decentralized nodes. This is supposed to open up in the future.

Mirrornet, whose nodes listen and report on mainnet transactions. Anyone can put up a mirror node. Mirror nodes can be general listeners, listening to all mainnet traffic, or they can be specialized to listen to just specific topics. Think of a news service that you could subscribe to.

Note: There are also test nets, which let you develop your apps. I’ll be working with those in a later post.

While you can send transactions and queries to mainnet nodes, you only send queries to mirror net nodes. Mirror nodes can specialize in a particular topic, which anyone can subscribe to. And last thing is, every node on either net is equal to every other node - that is, there is no leader node.

So that’s what I got out of the whitepaper. As I said, I felt like I needed more information.

Episode 29 of Gossip About Gossip

Fortunately after a small bit of hunting around I found Episode 29 of the Gossip about Gossip podcast. In less than half an hour, Paul Madsen and Ken Anderson filled in the blanks. I’ll describe what I learned in a nutshell.

The Why of HCS

The biggest advantages of Hedera is its ability to guarantee a final ordering of all messages immediately, and that it has a very high tolerance against network faults and malicious interference. Hedera started out with this consensus facility embedded in the hashgraph. When they realized that people needed a way to confirm messages as being true, efficient, and economical, they decided the best way to provide that was to surface the hashgraph consensus layer in a service. Though it’s possible to create a consensus service using smart contracts, a file system, or even using cryptocurrency service messages, the hashgraph consensus layer was the only option that qualifies on the true, efficient, and economic fronts.

In both the documentation and in the video, they speak about HCS’s ability to enforce fair ordering. I like to think of it more as disinterested -- due to it’s fault tolerant nature, the consensus service can’t give any preference to any one transaction over another. Gossip announces messages, and gossip about gossip determines ordering. They call it fair. Whatever.

Using HCS

Ken and Paul talk a bit about the likely use case of a company having a private Hyperledger service, needing to use something like HCS to determine transaction ordering, but not wanting to jeopardize proprietary information through a public network like Hedera. The solution is to either hash the message or encrypt it. In distributed applications you might also hash/encrypt the portion of code that operates on the private information, thus guaranteeing that all listeners wind up with the same final messages.

Ok, So What’s In It For Me (As a Developer)?

If you are looking to be a software engineer, you’re going to need to use the HCS. At minimum, you’ll need to know how to:

• Send a plaintext, hashed, or encrypted message to the HCS mainnet. You’ll probably need to know how to send messages per some schedule.
  
• Query both the mainnet and mirrornet for a specific topic.
• Query a specific node or set of nodes on either net. 
• Mange the dehashing or decrypting of query results.
  
• Execute some action on the basis of query results.

I’ll be going through those specific tasks down the road.

If you are feeling adventurous (or if your company/application requires it), you may also want to know how to

• set up a mirror node that listens for messages that specify a specific target

• manage the mirror node for security and subscriptions

I will likely look into these as well, though a bit further down the road.

Stopping Point

Here I pause my investigation into the HCS. Things are starting to come together. I’m going to take a look at the Cryptocurrency service next. I suppose I could have looked at it before digging into the HCS, but as I’m definitely going to need to understand the Cryptocurrency service before looking into Smart Contracts, I decided that things will be fresher in my mind with this order.

Foot note

In the video Ken and Paul spend time talking about Hyperledger Fabric. Hyperledger Fabric is an effort driven by the Linux Foundation. The article, “Who Invented Hyperledger,” on the 101 Blockchains web site provides a nice and succinct overview of the Hyperledger project. I also like their Hyperledger Fabric Tutorial, which goes in just deep enough technically to give you an idea of how you use it.