Maximizing analytics performance III -- SAP's HPA

Top-tier general-purpose relational DBMS -  which for me is a long way of saying  "Oracle, DB2, in some cases SQL Server and/or Sybase Adaptive Enterprise, and perhaps also the old Informix products IBM bought (but I haven't really kept up with those)"  -- contain a lot of clever techniques for accelerating analytical performance.  But while they provide a good-enough solution for most purposes, sometimes a more specialized out-of-the-box trick can blow them away.

One very promising approach is SAP's HPA (High Performance Analytics), part of NetWeaver.  Actually, it's an entirely-in-the-box trick; they put the whole thing in main memory.  When they do that, indexing becomes moot; queries are just resolved via full table scans, with very fast and consistent response times.

The extent of the usefulness of this approach depends upon how far they can push two technologies.   One is the Infocube architecture, augmented with some extra compression.  The other is just highly-parallel processing.  HPA is optimized for blade servers in partneship with Intel, with HP being the hardware vendor that’s – no, I will not say “first out of the box.”  

Why does this story depend on blades?  For the very simple reason that it depends on RAM -- more blades ==> more RAM ==> more data served.  The practical limit right now appears to be somewhat under a terabyte of RAM (8 gigabytes/blade x 100+ blades), but that can be enough to handle large-enough pieces of a data warehouse with multiple terabytes of underlying data.