DataWarehouse with Ruby on Rails

Recently joined www.rubyforge.org. Wish to join a few projects on similar direction.

While browsing through I found a project they call “ActiveWarehouse” using Ruby on Rails. Pretty interesting for me, since I come from Warehousing background myself.

I guess, I am going to read more about it and probably join the project. See if I can contribute something.

Aggregator : Sorted Input

You would read everywhere that an aggregator responds faster if you provide it sorted data. Yeah, thats correct in theory.

But, be careful while doing that. If you decide to use Sorter transformation to sort data, the end result might be slower. However, another option is to use sorting at the source qualifier level..

In one of my experiments on Powercenter 7.1.3, I had the following stats :

Source Data : 277000 rows
time for an aggregator :

No sorted input : 22 seconds
With sorter in mapping : 50 seconds
With sorting in Src Qualifer : 16 seconds

My view is, if possible, do the sorting at db end.