Abstract:

To solve the existing problem of the decreasing low performance of the relational database caused by dead lock and blocking as the bulk data increase, the authors put forward a method of handling data with HDF5 and Esper by taking the advantage of HDF5's powerful I/O performance and Esper's high operational efficiency property. Thus, the design of a portable data format and a function bank may solve the problems of performance and efficiency that exist in the instant systematic storage, management, exchange and the filing requirement of the data analysis. The experimental results indicate that HDF5's data reading and writing performance are 80 times higher than that of the MSSQL under the condition of 10G per day data bulk. Taking Esper's millisecond level handling capacity into account, the system has a  relatively high application value in the field of data handling analysis and the realtime requirement.

Key words: relational database；HDF5；effective data analysis system；analytic model for foreign exchange