[Bizgres-general] Question: Is 64-bit cache support valuable?

[Simon Riggs]

On Wed, 2005-07-20 at 10:26 -0500, Jim C. Nasby wrote:
> On Wed, Jul 20, 2005 at 12:38:39AM +0100, Simon Riggs wrote:
> > 8.0 uses a scan resistant buffer strategy. 8.1 does not, IIRC. I'm not
> > sure that having a large shared_buffers works well in either case. In
> > the former, scans won't put data in the cache, in the latter a scan of a
> > large table will flush much of the cache out. 
> 
> Ugh. I thought the new buffer stategy in 8.1 was scan resistant like 8.0
> was? ISTM having a buffer that's resiliant to seqscans will be pretty
> important for any kind of DW application. How horrible would it be to
> have a different or additional buffer management strategy?

I'll check.

> Also, something that's always irked me is how sorts are handled. The
> problem is that unless you know your workload really, really well, you
> can't give large sorts as much memory as they'd like to have without a
> serious risk of running the system out of memory. Is there something
> that can be done to put a global limit on sort memory (and possibly
> other memory uses)? The idea I've proposed in the past is that as each
> sort consumes or releases memory, it updates a global counter that
> tracks how much sort memory is in use. As this approaches the set limit,
> sorts would be allowed to allocate less memory. Ultimately, the idea is
> to do everything possible to allow sorts that can fit entirely in memory
> to do so, while forcing sorts that obviously can't fit in memory to
> disk. So, like buffer management, how hard would it be to have
> different/additional sort memory management from the base postgresql
> code?

Almost exactly what I had in mind. Spooky, but good. I'll not discuss
the full design just yet, but it has the features you mention, plus some
extra control features.

Best Regards, Simon Riggs