In the paper Google Cloud Computing Platform Technology Architecture and the Impact of Its Cost, which is published by Jia Xiaojing, the author compares the architecture between Google Cloud Computing platform and traditional IT system.
Data Storage Technology
Core technical indicators: storage reliability, I / O throughput and scalability.
Google developed a distributed file system based on several assumptions:
Hardware failures are the norm.
Supporting large data sets.
Processing mode of write once and read many.
High concurrency.
To ensure reliability of data, Google File System uses a redundant storage means.
GFS's writing operation separates control signal from data flow.
Data Management Technology
Google developed a large-scale database system Big Table with the weak consistency requirements.
Big Table system depends on the underlying structure of the cluster system, which includes a distributed cluster task scheduler, the GFS which has been addressed above, and a distributed lock service Chubby.
Big Table's main server not only manage the metadata but also is responsible to manage remotely and allocate for the Tablet Server.
Programming Model
Map Reduce is not only the programming model that processes and brings about a large data set but also an efficient task scheduling model. (The programmer can finish the distributed parallel program development that specifies the data processing to each block data in the Map function and how to regress the intermediate results of the block data processing in the Reduce function)
Five steps to implement Map Reduce program:
1. input file
2. assign one file to many parallel workers
3. write intermediate file (local writing)
4. reduce workers running
5. output the final result
In Map Reduce model, because data is managed dispersively by each node, there is no separate, centralized database system, and each node can only operate the data managed by it. Consequently it needs the intervention of upper application software to ensure consistency and integrity of across-node data.
The system in the Map Reduce mode increases the map that decomposes the task, the Reduce other processing link about the protocol of results, and parallel processing to support multiple worker node, while need to complete the failure handling about worker node, as well as coordination and communication between the worker nodes and so on.
Cost Analysis of Google Cloud Computing Platform
Due to the distributed data storage and data management, Google reduces the capacity requirements on a single node.
It is easy to achieve standardization of equipment because all nodes are isomorphic.
Google Cloud Computing platform takes hardware failure as the norm and turns to switch automatically between nodes to achieve high availability by means of software fault-tolerant, significantly reducing equipment redundancy.
Based on the unique advantages of parallel computing, Google has developed excellent load balancing technology that enable to ensure business continuity in the means of dynamic load switching in different data centers worldwide.
Conclusion
3 Basic Characteristics of Google Cloud Computing Platform Architecture
1. system built on large-scale clusters of cheap servers
2. build synergies between infrastructure and the upper application (to achieve maximum efficiency in utilization of hardware resources)
3. achieve fault-tolerant nodes through software
Google applies Top-Down design method, namely, from the upper application, the infrastructure is reconstructed based on the operational characteristics about the specific application (rather than the optimization of a general sense)
No comments:
Post a Comment