March 6, 2014 Leave a comment
Allen Wei , Infosphere Warehouse/DB2 LUw , SVT, IBM
DB2 with BLU Acceleration is a state of art columnar store RDBMS (Relational Database Management System) master piece that combines and exploits some of the best technologies from IBM and Intel. In the video linked below, there is mention of an 88x speedup when compared with the previous generation of row store RDBMS on the exact same workload. That announcement was made during IBM IOD in November 2013.
Guess what? In a test done a few days ago (less than 3 months after the video was filmed), the speedup, again comparing DB2 with BLU Acceleration with row store RDBMS using the exact same workload on new Intel Xeon IVY-EX based hardware, is now 148x. Really? Need I say more? This shows that not only is DB2 with BLU Acceleration equipped with innovative technologies, but it also combines the exact set of technologies from both RDBMS and hardware advancement that you really need. This helps BLU Acceleration to fully exploit hardware capacities to the extreme and to give you the best ROI (Return on Investment) that every CTO dreams about.
You might start wondering if this is too good to be true. I have shown you the numbers. So, no, it is the truth. You might want to ask, even if this is true, is it complicated? Well, it does take discipline, innovative thinking and effort to offer technologies like this. However, my answer is again a No. It’s completely the opposite! In fact, As Seen on TV (the video clip), it’s as simple as – create your tables; load your data and voila!! Start using your data. There is no need for extensive performance tuning, mind-boggling query optimization or blood-boiling index creation. Leave these tasks to DB2 with BLU Acceleration. Can’t wait to try for yourself? It really is that fast and simple.
Do you need to hear more before you are 100% convinced? Let me begin by recalling a few key disruptive technologies that are built into DB2 with BLU Acceleration. This is mentioned in the video clip as well, and I will prove to you that we are not listing for the sake of listing them.
What state of the art technology was built into DB2 with BLU Acceleration that makes it so great? Here is a summary of what you saw on in the video clip:
# Dynamic In-Memory Technology – loads terabytes of data into random access memory instead of hard disks, streamlining query workloads even when data sets exceed the size of the memory.
- This allows the CPU to operate efficiently without waiting on the disk I/O operations
- In my case, for one instance, I could fit 2TB database into 256GB RAM or 1/8 of the database size
- I could also fit a 10TB database into 1TB RAM or 1/10 of the database size, in another test.
# Actionable Compression – Deep data compression and perform actions directly on uncompressed data
- Deep compression
- I noticed a storage space consumption that was 2.8x – 4.6x smaller than corresponding row-store database, depending on the size of the database
- Data can be accessed as is in the compressed form, no decompression needed
- CPU can dedicate power to query processing not on decompression algorithms.
# Parallel Vector Processing – Fully utilize available CPU cores
- Vector is processed more efficiently hence there’s an increase in CPU efficiency
- All CPU cores are fully exploited.
# Data Skipping – Jump directly to where the data is
- We do not need to process irrelevant data.
Have you been convinced, yet? I know you have. However, you don’t need to just take my word for it. Try it. The time you spent on reading the blog and trying to find a loophole is enough to give yourself a high performance database from scratch. Period.
Read more about the Intel and BLU Acceleration partnership here : DB2 BLU Acceleration on Intel Xeon E7v2 Solutions Brief
Allen Wei joined IBM as a developer for the BI OLAP product line, including OLAP Miner. He was a key member of the Infosphere product line, and has lead Infosphere Warehouse and DB2 LUW SVTs. Currently he focuses on tuning the performance of BLU Acceleration, mainly w.r.t the Intel partnership.
Visit the IBM BLU HUB to learn more about the next gen in-memory database technology!
Also checkout this post on the Intel Blog about how IBM and Intel have been working together to extract big data insights.