I would agree – 125,000 column table might be a bit of a burden. =)

I’ve not heard of the EAV model, nor seen it, but I may have to do some research to see what it is all about.

Just guessing but I suppose that 11g unpivot would be useful if one encounters data in a column_tab model that must be translated to rowtab.

Have you encountered the EAV Entity-Attribute-Value data model? That’s the model where each row contains data and metadata – it contains both the values and names of data ( the joins are horrible to look at). Supposedly this model is good for lab experiments e.g. mass screening, where – over simplifying – no two assays have the same shape. So it would be possible to create a new table for each assay or else use EAV and put all the assays and their results in one table. I worked with such a application before I knew the model had a name. In 1994 we weren’t convinced that it was the best way to store experimental data. I switched over to retail in ’96, so I don’t know they changed the model since then.

Also consider this: How would you partition a column-wise pivot table? How could you possibly index it? You really cannot. The row-wise design lends itself very well to partitioning. One could easily partition by list on the PRODUCT column or even partition by date range/interval on the RECENCY_TS column, or both.

The other point that I did not dive into was loading. The PL/SQL code that loaded the original pivot table was cursor loop based and it did not scale. It also did not support having more than one date for any product for a given customer. At this site I was able to load the row-wise table and generate a matching pivot table orders of magnitude faster than the PL/SQL could load the pivot table.

The problem with such designs is that it severely limits the usefulness of that data, as queries that were not known at the time of design often time become problematic.

Thank you for posting these examples. When designing databases for clients, our biggest challenge tends to be the client changing their mind about business rules, often and unexpectedly. Understanding that it happens helps us build databases defensively so they can take the strain of completely different approaches.

Also, it’s great to see an example of pretty complicated business logic represented in more-or-less “standard” SQL so it’s portable between Oracle and other RDBMS. (Non-Oracle peeps tend to complain that Oracle’s features are a hinderance.)

I second your points in general, but I think one important aspect of the column-wise storage is that you can gather statistics on these columns and therefore potentially get more accurate selectivity/cardinality estimates of the cost based optimizer.

If your example was part of a larger query then the accuracy of the cardinality estimate can be very important for the overall plan generated.

Although there are cases where you can alleviate this by using dynamic sampling I don’t think that it’s going to help in this particular case since you don’t have a single table predicate when using the pivot approach.

So as always it depends and one has to weigh flexibility and assess performance of the different approaches for each particular and individual situation.

Regards,
Randolf