Blockchain developers—Microsoft, IBM, the London Stock Exchange, and others—are fine-tuning their technology to prepare for the coming mainstream adoption outside the cryptocurrency niche. Along with massive redundancy and encryption benefits, blockchain also has a few areas that need improving. One problem shared even by the more polished coins, namely Bitcoin and Ethereum, is low throughput due to the massive size.
Because of the decentralized nature of the blockchain [distributed ledger], which spreads across expansive swaths of Internet real estate, data encryption tends to lumber along like a gigantic whale.
Though Blockchain is highly secure, it is hashing monster amounts of information in one swallow. Its decentralized construction means that data must travel to far-flung server destinations before it can turn over a single transaction—a rate of roughly 14-15 per second.
That’s nowhere near comparable to centralized payment companies like Mastercard®. As it is, all calculations being performed on Ethereum have to be performed on each node system-wide.
That is too unwieldy to meet increasing data use demands for a worldwide community. There is a solution, though, and it’s something that data science has already implemented. It’s called sharding.
Sharding is the process of horizontal partitioning of data when accessing very large databases or search engines. It divides the data into smaller, faster and more easily managed subsets, making the process more manageable for the user.
Each individual partition is called a shard or database shard and is evenly distributed or stored in separate database servers to spread the load. Without sharding, the regular transaction validation that occurs over the blockchain is not ideal.
That’s because the validation process, the one that confirms money only gets spent once, depends upon consensus algorithms.
Blockchain before sharding | a validation needs proof of work
A consensus algorithm, like bitcoin's proof of work (POW), does two things.
POW confirms that each consecutive block in a blockchain is the one version
POW keeps hackers from derailing the system and the chain from being forked
In proof of work, miners compete to add the next block (a set of transactions) to the chain by solving a cryptographic puzzle. The first to solve it receives 12.5 newly minted bitcoins – and a small transaction fee.
Critics of this process argue that the amounts of computational energy required to solve these puzzles and the residual heat generation are inefficient at best and damage the environment.
Without sharding, everyone in the network has to validate a transaction, costing them money. “Everyone” refers to those who mine. The resource consumption is extremely high and too much bandwidth gets “eaten up.”
That’s a big price to pay for only 14 transactions per second. Sharding changes the framework by virtualizing.
Blockchain after sharding | a validation needs proof of stake
In proof of stake (POS), people stake ether or money.
A POS-type of consensus algorithm doesn’t require investment in expensive computer equipment. Instead, “validators” invest in the coins of the system.
Because no coin creation (mining) exists in proof of stake, participants are validators. All coins exist from day one. Validators or “stakeholders” are paid transaction fees instead.
With sharding, much smaller groups need to validate.
There is a slight tradeoff in security, but by reducing the amount of data throughput, the whole process speeds up (performance and efficiency increase as well). Virtualization reduces the amount of work necessary to generate a new block.
Sharding makes sense on many levels, most especially in the ability to increase transaction rates in a field that is getting ready for a whale of a ramp-up.