[Npgsql-devel] PostgreSQL COPY functionality

Kalle Hallivuori kato at iki.fi
Fri Sep 21 20:44:07 UTC 2007


Hi!

I'm going to implement support for PostgreSQL COPY functionality in
Npgsql on behalf of truviso.com. I recently made the same for the
PostgreSQL JDBC driver: http://kato.iki.fi/sw/db/postgresql/jdbc/copy/

I have glanced at the (quite neat) sources of Npgsql. This is my
current take on design:

I will only implement raw copy data passing between client and server
for now. Any wrappers for the streams to handle encodings and object
serializations can be done on top of this later. I have chosen streams
as presentation method of choice to support easy exchange between
server and local files, but other approaches (ResultSet?) are possible
as well.

I need to find the correct place to publish setters and getters for a
StreamWriter and StreamReader. The associated stream operator objects
would be stored in Mediator or Context to be used for passing the copy
data between user and NpgsqlCopyStates.

Modify Mediator (or Context?) to hold either of StreamWriter
CopyStreamWriter or StreamReader CopyStreamReader for COPY operations.
These can be set by user before sending COPY statement or internally
by NpgsqlCopyState.StartCopy implementations. Having both set is an
error.

Modify following NpgsqlState methods:
- ProcessBackendResponses(): enable nonblocking calls to only handle
pending notifications (say, with adding an optional parameter?)
- ProcessBackendResponse_Ver_3(): handle COPY messages as follows:
  a) CopyInResponse, CopyOutResponse: change context.CurrentState to
instance of corresponding CopyState and call its StartCopy() method;
return if it returns a true boolean value.
  b) CopyData: call self.ReceiveCopyData() to handle it
  c) CopyDone: call self.CopyDone()
  d) CopyFail: call self.CopyFail()
  e) check that CommandComplete accepts "COPY"

abstract NpgsqlCopyState : NpgsqlState
- Represents Copy sub protocol state of server connection.
- Declares StartCopy, the copy operation initializer.

NpgsqlCopyInState : NpgsqlCopyState
- StartCopy(): have CopyStreamReader? SendCopyData( input from
CopyStreamReader ) until end of stream, then EndCopy() and return
false to return to user; otherwise set CopyStreamWriter to a custom
implementation that calls SendCopyData()
- SendCopyData(): send given piece of data to server, call
ProcessBackendResponses() without blocking to handle any pending
notifications.
- EndCopy(): send CopyDone to server, handle results with call to
ProcessBackendResponses()
- FailCopy(): send CopyFail to server, handle results with call to
ProcessBackendResponses() -- TODO: Figure out where to call this from
and when.

NpgsqlCopyOutState : NpgsqlCopyState
- StartCopy(): have CopyStreamWriter? write to it data read with
GetCopyData() until CopyDone; return false to return to user;
otherwise set CopyStreamReader to a custom implementation that calls
GetCopyData() to fetch each next piece of data.
- GetCopyData(): call ProcessBackendResponses() to retrieve a piece of
data to return.
- ReceiveCopyData(): called from ProcessBackendResponses() to set a
piece of data to be returned by GetCopyData().
- CopyDone(), CopyFail(): nothing much to do here if NpgsqlState
provides user with correct status information.

I would appreciate any comments before I start writing code.

I tried to compile the project with Mono 1.2.5.1 on OSX. Got the blurt
below. Insight appreciated. I can resort to Linux or Windows + VS if
required.

      [csc] Compilation succeeded - 55 warning(s)
                   [al] Compiling 6 files to
'/Users/kato/prj/npgsql/Npgsql2.study/src/build/ms/de/Npgsql.resources.dll'.
                   [al] **
(/Library/Frameworks/Mono.framework/Versions/1.2.5.1/lib/mono/1.0/al.exe:1482):
WARNING **: Missing method .ctor in assembly
/Users/kato/prj/npgsql/Npgsql2.study/src/build/ms/Npgsql.dll, type
System.Runtime.CompilerServices.RuntimeCompatibilityAttribute
                   [al] **
(/Library/Frameworks/Mono.framework/Versions/1.2.5.1/lib/mono/1.0/al.exe:1482):
WARNING **: Can't find custom attr constructor image:
/Users/kato/prj/npgsql/Npgsql2.study/src/build/ms/Npgsql.dll mtoken:
0x0a0001c3
                   [al] Unhandled Exception:
System.MissingMethodException: Method not found:
'System.Runtime.CompilerServices.RuntimeCompatibilityAttribute..ctor'.
                   [al]   at <0x00000> <unknown method>
                   [al]   at (wrapper managed-to-native)
System.MonoCustomAttrs:GetCustomAttributesInternal
(System.Reflection.ICustomAttributeProvider,System.Type,bool)
                   [al]   at
System.MonoCustomAttrs.GetCustomAttributesBase
(ICustomAttributeProvider obj, System.Type attributeType) [0x00000]
                   [al]   at
System.MonoCustomAttrs.GetCustomAttributes (ICustomAttributeProvider
obj, System.Type attributeType, Boolean inherit) [0x00000]
                   [al]   at
System.MonoCustomAttrs.GetCustomAttributes (ICustomAttributeProvider
obj, Boolean inherit) [0x00000]
                   [al]   at
System.Reflection.Assembly.GetCustomAttributes (Boolean inherit)
[0x00000]
                   [al]   at Mono.AssemblyLinker.AssemblyLinker.DoIt
() [0x00000]
                   [al]   at
Mono.AssemblyLinker.AssemblyLinker.DynMain (System.String[] args)
[0x00000]
                   [al]   at Mono.AssemblyLinker.AssemblyLinker.Main
(System.String[] args) [0x00000]

BUILD FAILED - 0 non-fatal error(s), 72 warning(s)

External Program Failed:
/Library/Frameworks/Mono.framework/Versions/1.2.5.1/lib/mono/1.0/al.exe
(return code was 1)


Cheers,

-- 
Kalle Hallivuori +358-41-5053073 http://korpiq.iki.fi/



More information about the Npgsql-devel mailing list