[Bizgres-general] Sort performance improvements..

[Ayush Parashar]

All,

Recently some performance enhancements related to external sort have been
made in bizgres, that has dramatically improved the sort for really large
tables. Unlike the previous versions, the new external sort code for bizgres
fully utilizes the work_mem allocated to the session. More the work_mem,
better the performance. In addition to better utilization of work_mem, the
new code also utilizes resources optimally to enhance external sort even at
low work_mem.

Simon, can you please throw some more light on the enhancements made..

I have included some performance test results for external sort that proves
the dramatic improvements.

Query: 
select count(*) from (select L_PARTKEY from lineitem order by L_PARTKEY) as
a;

Time taken in seconds for work_mem in MBytes:
------------------------------------------------------------------------
work_mem         Bizgres 0.9        Bizgres 0.8
------------------------------------------------------------------------
16            788.9            1176.7
32            531.8            986.7
64            439.4            931.9
128            383.5            912.1
256            391.0            924.7
512            407.4            1011.3
------------------------------------------------------------------------

About performance results environment setup:
Machine: 64bit Dual CPU AMD Opteron, 2191.435 MHz, 2GBytes of RAM
running Redhat Enterprise Linux (2.6.10-1.741_FC3.rootsmp)
Tables used: lineitem
Data generated using dbgen for TPC-H scale factor of 10.

This improvement will help enhance the performance of large class of
data-warehousing queries that depends on external sorting.

Ayush