Skip to main content

Module copy_function

Module copy_function 

Source
Expand description

Copy function registration (DuckDB 1.5.0+).

Extensions can register custom COPY TO handlers that define how data is exported to a specific file format. The lifecycle consists of four phases:

  1. Bind — inspect the output columns and configure the export.
  2. Global init — set up global state (open file, allocate buffers).
  3. Sink — receive data chunks to write.
  4. Finalize — flush and close.

§Threads

The data pointers this API passes around are untyped, so the compiler cannot check this for you. Treat each one as &T shared between threads:

  • extra_info (CopyFunctionBuilder::extra_info) lives as long as the database and is read from every connection’s thread: T: Send + Sync.
  • Bind data (CopyBindInfo::set_bind_data) is read by every sink call. COPY … TO with PER_THREAD_OUTPUT or PARTITION_BY runs the sink on several threads at once, all sharing the one bind data: T: Send + Sync, and do not mutate it without a lock.
  • Global state (CopyGlobalInitInfo::set_global_state) is one per output file. With PER_THREAD_OUTPUT each thread gets its own; with PARTITION_BY a partition’s state can be reached from more than one thread; with neither option the sink is not run in parallel. Mutating it through the raw pointer is only sound under a lock (Mutex) unless you know neither option is in play, so make it T: Send and guard mutation.

All three may be dropped on a different thread from the one that created them.

§Example

use quack_rs::copy_function::CopyFunctionBuilder;

let builder = CopyFunctionBuilder::try_new("my_format")?;
// .bind(my_bind_fn)
// .global_init(my_init_fn)
// .sink(my_sink_fn)
// .finalize(my_finalize_fn)

Re-exports§

pub use info::CopyBindInfo;
pub use info::CopyFinalizeInfo;
pub use info::CopyGlobalInitInfo;
pub use info::CopySinkInfo;

Modules§

info
Callback info wrappers for copy function callbacks.

Structs§

CopyFunctionBuilder
Builder for registering a custom COPY TO function.

Type Aliases§

CopyBindFn
Callback type aliases for copy function phases.
CopyFinalizeFn
Finalize callback — called once per output file, after its last sink.
CopyGlobalInitFn
Global init callback — called once per output file to set up its state.
CopySinkFn
Sink callback — called for each data chunk to write data. With PER_THREAD_OUTPUT or PARTITION_BY it runs on several threads at once; with neither, not in parallel (see the module docs).