[Bizgres-general] Ketl functionality in bizgres

[Nicholas E. Wakefield]

Hi Alon,

Some answers for you.

>> 5. Enhancement: The beta PGBulkWriter has been added.
>1 - Is it using a JDBC postgres COPY interface? If so, is it in an
official
>JDBC driver release or is it a patch to JDBC that implements access to
COPY
>(I seen one around in the last year or two).

It's the patch. Plus I made some very minor changes to it, to improve
exception handling. To patch the latest driver you'll need to modify the
patch a little or I can maybe produce a newer patch.

>2 - which options/enhancements does PGBulkWriter supply on top of the
JDBC
>driver?

3 Layers to this.
  1. The patch, exposes something called the copy manager via the
copyAPI metod. This allows copy commands to be send to the db via JDBC.
  2. PGCopyWriter, this sits on top of the copy manager to give it a
prepared statement feel. It deals with producing the command, escape,
data formats etc.. All you do is prepare, setParameters for each tuple,
submit it to the batch and then execute the batch.
  3. PGBulkWriter, this sits on top of the PGCopyWriter to expose the
the copy functionality t KETL. So by simply switching the class names in
the KETL xml, you start bulk loading using the copy command.

Note. The PGBulkWriter doesn't perform multiple inserts at once, using
threads, we have another component that does this but it's not ready for
release. This one works in a similar way to a ring bus.

>If this is a JDBC release feature, or something that you wrote that
works
>well we could integrate it in the java data loader potentially and
remove
>its jni/libpq layer. Any thoughts about it?

I would like to see this patch be part of the standard driver, as people
do ask for it every once in a while. If you want to integrate into the
loader feel free, I can talk you through how it works, its really
simple. Give me a call if you want to talk more - Luke has my number.