[Bizgres-general] Question: Is 64-bit cache support valuable?

[Jim C. Nasby]

*PING*

This hasn't gone to -hackers yet, right?

Actually, what made me think of this was a recent suggestion about
combining shared memory consuming stuff into one pool (I think it was
FSM). The idea was that FSM (or whatever the discussion was about) could
then be sized without a restart, which for something like FSM could be
quite useful.

So maybe the direction that PostgreSQL should head is that it has two
over-arching memory limits: one for shared memory and one for non-shared
memory. All shared memory objects would pull out of the shared memory
pool, with all left-over space being used for buffers (of course all
these other shared memory objects would still need limits on them so we
didn't run out of buffers). All other memory consuming
objects/operations, such as sort, would have to obey a total memory
limit, designed to ensure that the machine doesn't swap. ISTM that
whatever memory we're not using up to that limit could be considered the
sort pool.

On Thu, Jul 21, 2005 at 07:39:15PM +0100, Simon Riggs wrote:
> On Thu, 2005-07-21 at 11:28 -0700, Josh Berkus wrote:
> > Simon,
> > 
> > > The challenge is to come up with a scheme that has these features:
> > > - prevents overallocation of memory which results in swapping
> > > - maximises allocation of large memory servers
> > > - allows heavy memory users to "go large" when they need to
> > > - prevents low memory users from conflicting with heavy users/each other
> > > - allows the administrator to have control over the system
> > > - allows some level of control at user level e.g. "need lots, please"
> > > - must be easy enough to get right in real-world use, not benchmarks
> > > - also need to take into account temp_buffers settings
> > > - there may also be a requirement for predictability, so that important
> > > regular tasks can complete in known durations or by deadlines
> > 
> > Sounds good but too complicated for a first implementation.  
> 
> Sounds reasonable.
> 
> >  I'd narrow down 
> > the critical features to:
> > 
> > 1. Allow 64-bit work_mem (and maintenance_work_mem) allocation.
> > 2. Add a total_work_mem limit which would be tracked for work_mem, 
> > maintenance_work_mem, and temp_buffers.
> > 3. Add a system where processes requesting over the limit or total_work_mem 
> > would block and wait.
> > 
> > I think that would be enough code for a first, useful implementation and 
> > further improvements would be suggested by the users/results.   Adding the 
> > hooks for centralized tracking and management of allocated memory will be 
> > enough code as it is.
> 
> I'll move to -hackers for the more detailed aspects, once 8.1 beta is
> out.
> 
> Best Regards, Simon Riggs
> 
> _______________________________________________
> Bizgres-general mailing list
> Bizgres-general at pgfoundry.org
> http://pgfoundry.org/mailman/listinfo/bizgres-general
> 

-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby at pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461