Big data VS. Blockchain Part 1

Common keyword: Distributed, which means the transferring from technological monopoly to distributed technology.

Distributed storage: HDFS VS. Block

Big data, which should meet the requirements for the massive and rapid growth storage space, and it is a great cost for underlying hardware architecture and file system compared with traditional technology, then can be flexible for the expansion of storage capacity. Google’s GFS and HDFS has laid the groundwork for big data storage technology. In addition, there is another challenge for storage about the data formats. So the storage architecture of underlying storage layer is not just HDFS, as well as HBase and Kudu.

Blockchain, which is the underlying technology architecture for Bitcoin, and it is a kind of distributed ledger. As a sustained growth data structure with ordered blocks, all of the nodes involved in computing and recording through nework, to verify the efficiency of mutual  information. From this point, blockchain technology is a specific kind of database technology. Due to the characteristics of secutity and convenience for the distributed database, many industry insiders believe that it is the upgrade and supplement for the exsiting Internet technology and it will have a optimistic future.

Distributed computing: MapReduce VS. Consensus mechanism

Data mining is a kind of data intensive computing, which requires a large amount of distributed computing capabilities. And the key techbology is for nodes management, task scheduling, fault-tolerant and high reliability. MapReduce of Google and Hapoop is a representative of such distributed technology, and it can improve the scale out ability for the extened linear system by adding server noedes, to reduce the cost and enhance the scalability. Nowadays, in addition to batch computing, big data also includes the flow calculation, graph computing, real-time computing, interactive query and other computing framework.

Consensus mechanism of Blockchain is to reach a consensus among all the distributed nodes, to generate and update data through the algorithm, then to determine the effectiveness of a record, which represents a means of identification and a way to prevent tampering. Blockchain mainly includes four different consensus mechanisms, which can be suitable for different application scenarios, to reach a balance between efficiency and security. Take Bitcoin as an example, the Proof of Work (POW), it is possible to forge a record only by taking control of more than 51% accounting nodes in the whole network.