Big Tables in Relational Database

* Total RDB row Count = 10,000,000

* Get Data row count = 500~1,000 /each search

* Total Search Count >100

Case Study：Semiconductor Recipe Management

[Phase I] Poc

1. Understanding the comparison logic

Each Version of recipe is established as a tree to all history records. Each node in the tree is recorded as an item with a value. The comparison each time will traverse through the whole tree and mark out the difference.

2. Porting the current logic to HBase

With the acknowledgement of the current logic, the new application will be implemented with new data model which will base on the HBase. The Converter will transfer the original data to Hbase depends on different data model. The optimal Hbase would make better performance for data access.

3. Providing the performance report

With the new application based on Hbase, we provide the test report of the performance through the data access. The evidence with the test result shows the cost time for users to query the version comparison through the new application. This helps user to review the improvement in fact.

4. Providing the total solutions.

The environment would be one of the critical part during the whole case. We provide the best suggestion and for the customers to establish their system, including the Hadoop, HBase Server and the system configurations which can provide the optimal and total solution.

5. Evaluation and Plan for integration

After the verification of the solution, we'll evaluate the process for customers to migrate their current data into server. This is important for the integration because the original data may contains lots of unpredictable situations. This make the uncertainty of the data migration increase. Also, the migration also need to provide the method for the coming new data to be going to the Hbase for further use. So the total integration may last long depends on the complexity of the data.

[Phase II] Integrate the original database

1. The original system data into HBase

2. Data can also provide a way to make new entrants into HBase