[Bizgres-general] Off-list Re: [ENG] Re: Statement Queuing take II - Resource Scheduling (Running with cost and cursors)

[Luke Lonergan]

> > Now memory is typically required upfront -  so time sharing 
> will force 
> > the machine to swap in all likelihood - possibly swapping *more* 
> > aggressively than if we let them all run together... 
> getting us back 
> > to why we wanted queuing in the first place... In the case 
> of IO, the 
> > time sharing may well flush the buffer cache for each query in turn 
> > and spoil any read-ahead that was working! However in the 
> case of cpu 
> > the time sharing will work perfectly - but the standard cpu time 
> > slicing algorithms handle this fine anyway!
> 
> Yep.

The purpose of having our own timeslicing scheduler instead of just
using the one built into the OS is to avoid the I/O conflicts that the
OS is not aware of, but we are.

Point by point:
- Postgres can and will only use O(1GB) of RAM per query within it's
process segment and we can establish the amount beforehand.  Therefore
we can control the number of runnables in the queues to ensure that
there will not be swapping.

- I/O readahead buffering is O(16MB) and is retained in the OS I/O cache
unless there is memory pressure.  Even if an I/O buffer is flushed, it
takes milliseconds to refill one, so our intra-queue timeslicing can be
chosen at a number like 30 seconds to ensure that we're not wasting
work.

Do these address the issues you identified?

- Luke