Research on possibility of RDBMS to have performance benchmarks like no-sql Databases (part 4)

We are continuing our discussion on new OLTP design from previous post.

( )

Why New OLTP Design Considerations?

Currently the choice is limited when it comes to OLTP selection (MySql , Sql Server , Oracle ) . These OLTPs are leaders for decades but we know the specialized DBs are beating them in order of two or three . New OLTPs are coming every month  in market with new designs and claims to beat present OLTPs and it will make our job harder when it comes to selecting OLTP vendor . The performance alone can never be a criterion what new OLTP vendors are claiming primarily . This is the reason we are researching a hypothetical OLTP which will have new designs . In the end of blog series we will create a cheat sheet which will analyze the current vendors in term of our new design expectations.

Today we will look into main memory.

Main Memory

In the late 1970’s a large machine had somewhere around a megabyte of main memory. Today, several Gbytes are common and large machines are approaching 100 Gbytes. In a few years a terabyte of main memory will not be unusual. Imagine a shared nothing grid system of 20 nodes, each with 32 Gbytes of main memory now, (soon to be 100 Gbytes), and costing less than $50,000. As such, any database less than a terabyte in size, is capable of main memory deployment now or in the near future.

The overwhelming majority of OLTP databases are less than 1 Tbyte in size and growing in size quite slowly. For example, it is a telling statement that TPC-C requires about 100 Mbytes per physical distribution center (warehouse). A very large retail enterprise might have 1000 warehouses, requiring around 100 Gbytes of storage, which fits our envelope for main memory deployment.

As such, we believe that OLTP should be considered a main memory market, if not now then within a very small number of years. Consequently, the current RDBMS vendors have disk oriented solutions for a main memory problem. In summary, 30 years of Moore’s law has antiquated the disk-oriented relational architecture for OLTP applications.

Although there are some main memory database products on the market, such as TimesTen and  olidDB, these systems inherit the baggage of legacy RDBMS as well. This includes such features as a disk-based recovery log and dynamic locking, which, as we discuss in the following sections, impose substantial performance overheads.

In next post we will look into Multithreading .