[Bizgres-general] Transactional Statement Queuing - Queue Design 1

[Mark Kirkwood]

Mark Kirkwood wrote:

Some second thoughts after a more complete reading of the code:

> 
> My initial reading of the code gives the impression that the light
> weight locks will do '1 holder, many waiters' (i.e. exclusive) or 'many
> holders' (i.e. shared) types of access, but not 'several holders, many
> waiters'. In addition the documentation implies that long wait
> activities should not use lightweight locks (tho I can't actually see
> any reason for that myself, as they wait on a semaphore).
>

While there are only two lock modes defined :'exclusive' and 'shared' 
(see src/include/storage/lwlock.h), it is fairly straightforward to either:

- add another lock mode (say 'share-limited' or just 'limited') that
   works like 'shared' but with a limit to the number of holders (or)
- amend the 'shared' section of code to:
    -> see if there is a limit associated with the particular lightweight
       lock
    -> set the mustwait flag if the number of shared holder is = the
       limit.

(this is discussing src/backend/storage/lmge/lwlock.c:LWLockAcquire)

The unlock code *may* be simplified in the case of the explicit new lock 
mode (i.e LWLockRelease), but any thoughts are welcome about which 
approach seems best.

> Finally it looks to me like the number of lightweight locks is 
> essentially fixed ( see NumLWLocks() ), which does not match our 
> requirement for a variable number of queues.
> 

While they are not *unlimited* in number, there is provision for 
creating additional ones, so adding a GUC (say 'max_queues' or similar) 
means enough of them can be created at server startup.

In conclusion - lightweight locks are back on the menu :-).

I'm in the process of coding up an initial cut of this mechanism, which 
should clarify some of these choices. Note that this is *only* the lock 
mechanism - attaching queue limits to roles and having the lock know 
what its limit is, are still TODO.

After I've got a working mechanism, I'm going to open the discussion to 
pg-hackers - to get some feedback on how I should *really* do it!

Cheers

Mark