[Bizgres-general] A bitmap index access method is available in bizgres CVS tip

[Simon Riggs]

On Thu, 2005-12-22 at 13:52 +1300, Mark Kirkwood wrote:

> I made a modification to generate Zipfian frequencies - however this did 
> not help that much (val2 index size decreased 600Mb -> 400Mb), I suspect 
> due to the data distribution (I've uploaded the modified version of the 
> test in case you guys want to try it out). So distribution looks to be a 
> very significant factor.

Yes. The less-uniform the more it compresses.

With such a distribution, if we add another 100 distinct values the
index will not be anywhere near twice the size.

> I definitely see Bizgres doing bitmap index creation faster than 
> Postgres 8.1.1 does the equivalent btree - for the *larger* indexes 
> (Postgres 8.1.1 is quicker for the smaller ones, but that is not really 
> significant).

Good

> Bizgres bitmaps are smaller than Postgres 8.1.1 btrees for the *larger* 
> indexes - Bizgres total index pages at approx 78000 vs Postgres 8.1.1 at 
> 150000. Notice the nationkey index is bigger in Bizgres, however this is 
> a very small index, so again, probably not significant here.
> 
> So for creation time and sizes I think I am seeing the same sort of 
> thing that your own testing did.
> 
> For query performance however, Postgres 8.1.1 is the same speed or 
> faster in all cases! Note that the filesystem was umount|mounted as well 
> as the databases restarted before running the queries, to ensure 
> retrieval from disk was tested. Also note that the count patch made 
> little if any difference.

Your testing looks excellent, as ever.

It wouldn't surprise me if some additional tuning of the executor was
needed to get things to live up to their full promise. Jie's done a
fantastic job of creating this new index AM in just months. Both GiST
and nbtree have taken a long time to get to where they are now.

Jie: great work.

Best Regards, Simon Riggs