pub struct CopyFunctionBuilder { /* private fields */ }Expand description
Builder for registering a custom COPY TO function.
All four lifecycle callbacks (bind, global_init, sink, finalize) should be
set before calling register.
Implementations§
Source§impl CopyFunctionBuilder
impl CopyFunctionBuilder
Sourcepub fn try_new(name: &str) -> Result<Self, ExtensionError>
pub fn try_new(name: &str) -> Result<Self, ExtensionError>
Creates a new copy function builder with the given format name.
The name corresponds to the format identifier used in
COPY table TO 'file' (FORMAT name).
§Errors
Returns ExtensionError if the name contains a null byte.
Sourcepub fn bind(self, f: CopyBindFn) -> Self
pub fn bind(self, f: CopyBindFn) -> Self
Sets the bind callback.
Sourcepub fn global_init(self, f: CopyGlobalInitFn) -> Self
pub fn global_init(self, f: CopyGlobalInitFn) -> Self
Sets the global init callback.
Sourcepub fn sink(self, f: CopySinkFn) -> Self
pub fn sink(self, f: CopySinkFn) -> Self
Sets the sink callback.
Sourcepub fn finalize(self, f: CopyFinalizeFn) -> Self
pub fn finalize(self, f: CopyFinalizeFn) -> Self
Sets the finalize callback.
Sourcepub fn copy_from(
self,
table_function: TableFunctionHandle,
) -> Result<Self, ExtensionError>
pub fn copy_from( self, table_function: TableFunctionHandle, ) -> Result<Self, ExtensionError>
Attaches a table function that implements COPY … FROM for this format.
Without it a copy function is export-only: COPY t TO 'f' (FORMAT mine)
works and COPY t FROM 'f' (FORMAT mine) reports that the format does
not support reading.
§How DuckDB drives it
The table function is not called the way a FROM my_func(...) one is.
CCopyFromBind synthesises the call:
- the file path arrives as positional parameter 0, a
VARCHAR; - every option in
COPY … FROM 'f' (FORMAT mine, X 1, Y 'z')arrives as a named parameter. An option the table function did not declare is a bind error, so declare each one withnamed_param; - those options arrive uncast.
CCopyFromBindstores each option’s value as written, not cast to the declared type:SKIP 'abc'reaches aBIGINT-declaredskipas theVARCHAR'abc', andSKIP 3as anINTEGER. An option written without a value (… , FLAG)) is not passed at all, so it cannot be told from an absent one. Check the value’stype_idbefore trusting it: anas_i64_or(0)on'abc'quietly yields the default; - the target table’s schema is already fixed, because
COPY … FROMloads into an existing table.duckdb.his explicit that the bind callback “should not define its own result columns usingduckdb_bind_add_result_column” and should read the expected schema fromBindInfo::result_column_countand its siblings instead. Nothing enforces that:DuckDBappends a declared column to theINSERT’s own list of expected types, so its chunks are wider than the table. A releaseDuckDBdrops the extra column; one built with assertions invalidates the database (docs/upstream-duckdb-reports.md, item 37). A typed reader (with_state) that declares one fails its bind; a raw bind callback must not.
DuckDB copies the table function here, so the handle may be dropped
immediately afterwards.
§Name the reader after the format
A bad COPY … FROM option is reported as
'X' is not a supported option for copy function 'NAME'. X is the
option as the query spelled it up to DuckDB 1.5.x, and lower-cased
from 2.0.0, whose parser lower-cases every COPY option name
(transform_generic_copy_option.cpp); the match against the declared
named parameters is case-insensitive on both. NAME
is the table function’s name, not the copy function’s:
CCopyFromBind reads info.tf.name, and
duckdb_copy_function_set_copy_from_function only substitutes the copy
function’s name when the table function has none — which
TableFunctionBuilder never
produces. So a reader called my_format_reader attached to a format
called csv2 tells the user about csv2’s options under the name
my_format_reader. Name the two alike.
§Errors
duckdb.h states the precondition plainly — “the table function must
take a single VARCHAR parameter (the file path)” — and nothing enforces
it. duckdb_copy_function_set_copy_from_function never looks at
tf.arguments beyond rejecting INVALID types, and CCopyFromBind
builds the argument list itself: it pushes exactly one Value holding
the path, whatever the function declared. So a reader that declares two
parameters, or one that is not VARCHAR, does not fail here — it fails
later, inside its own bind callback, reading an argument that is not what
it asked for. That is what this check turns into an error you get at
registration time.
The other preconditions need no check here: DuckDB silently does
nothing if the table function is missing its bind, init or scan callback
— which
build_handle
already refuses to produce — or if an argument or named-parameter type
contains INVALID anywhere, including nested inside a LIST or
STRUCT, which the builders reject at every reachable entry point (see
TypeId::is_composite). Only a
type built from a raw handle could still smuggle one through.
§Example
use quack_rs::copy_function::CopyFunctionBuilder;
use quack_rs::table::TableFunctionBuilder;
use quack_rs::types::TypeId;
// SAFETY: the callbacks match their declared signatures.
let reader = unsafe {
TableFunctionBuilder::new("my_format_reader")
.param(TypeId::Varchar) // the file path
.named_param("skip_rows", TypeId::BigInt)
.bind(bind)
.init(init)
.scan(scan)
.build_handle()
}?;
let _copy = CopyFunctionBuilder::try_new("my_format")?.copy_from(reader)?;Sourcepub unsafe fn extra_info(
self,
data: *mut c_void,
destroy: duckdb_delete_callback_t,
) -> Self
pub unsafe fn extra_info( self, data: *mut c_void, destroy: duckdb_delete_callback_t, ) -> Self
Attaches arbitrary data to this copy function.
Available in every callback via get_extra_info. The allocation belongs
to the builder until register hands it to DuckDB;
a builder that is dropped instead runs destroy itself rather than
leaking it.
§Safety
datamust be a pointerdestroycan free exactly once.destroymust not panic: it is anextern "C" fn, so an unwind out of it aborts the process at its own boundary. Usecatch_ffi_panicinside it.
Sourcepub fn extra_info_typed<T: Send + Sync + 'static>(self, value: T) -> Self
pub fn extra_info_typed<T: Send + Sync + 'static>(self, value: T) -> Self
Attaches value as this copy function’s extra info: the safe, typed
form of extra_info.
quack-rs boxes value behind a type tag and generates its destructor,
which contains a panic in T::drop and runs once — when DuckDB
releases the function, or when this builder is dropped unregistered.
Read it in any callback with extra_info_ref (on
CopyBindInfo and the other three
info types), which returns None for any other type instead of
reinterpreting memory. T is Send + Sync because every sink thread
reads the same value.
For per-statement state, see set_bind_data_typed and
set_global_state_typed on the info types.
Sourcepub unsafe fn register(
self,
con: duckdb_connection,
) -> Result<(), ExtensionError>
pub unsafe fn register( self, con: duckdb_connection, ) -> Result<(), ExtensionError>
Registers the copy function on the given connection.
A copy function may implement either direction, or both:
| Direction | Requires |
|---|---|
COPY … TO | bind + sink + finalize (and optionally global_init) |
COPY … FROM | copy_from |
A read-only format sets only copy_from and leaves the writing
callbacks unset — duckdb_register_copy_function accepts that, because
it decides what a copy function supports from
info.sink != nullptr and copy_from_bind != nullptr independently.
§A format name can be registered once
duckdb_register_copy_function uses ALTER_ON_CONFLICT, and for a copy
function that means the new one is dropped while the call still
returns success (DuckSchemaEntry::AddEntryInternal). A second
registration of my_format, or a format called csv, would silently
leave the existing function in charge. register therefore refuses a
format name that already resolves.
There is no catalog view of copy functions, so the check asks the
binder: it runs COPY (SELECT <missing column>) TO '' (FORMAT '<name>').
DuckDB resolves the format before binding the query, so a binder
error (the column) means the format exists, and a catalog error
means it does not; nothing is ever written, and no copy callback runs.
A format name that belongs to an autoloadable extension (parquet,
json, …) may be autoloaded by that lookup, exactly as the same
COPY typed by a user would; if that extension is then present, the
name counts as taken. Copy functions registered through the C API live
in the in-memory system catalog and never persist, so a name that
resolves is always a live conflict.
The probe is a statement that fails, and DuckDB 2.0 aborts the
transaction a failed statement runs in. On the entry point’s
registration connection, which runs the closure in a transaction
(Registration::Transactional), it
therefore runs on a short-lived connection of its own, and a format
registered earlier in the same transaction (which that connection
cannot see) is refused from the transaction’s own record. On any other
connection it runs on con, so calling register on a connection of
yours inside a transaction you opened aborts that transaction on
DuckDB 2.0.
§Errors
Returns ExtensionError if neither direction is implemented, if
COPY … TO is only partly implemented, if the format name is already
taken (or the check cannot tell), or if registration fails.
§Safety
con must be a valid, open duckdb_connection.