Expand description
Closure-based table functions with typed scan state.
This module provides TypedTableFunctionBuilder, a higher-level builder
layered on top of TableFunctionBuilder. It lets extension authors write
table functions using safe Rust closures instead of unsafe extern "C" fn
trampolines for bind, init, and scan.
§Motivation
The raw TableFunctionBuilder API requires authors to write three
hand-rolled unsafe extern "C" fn callbacks and manually shuttle state
through FfiBindData /
FfiInitData. For extensions that merely need
“take some parameters at bind, stream rows until exhausted”, that ceremony
is largely accidental complexity. TypedTableFunctionBuilder collapses
it to closures.
§Two entry points
| Constructor | Bind produces | Each execution starts from |
|---|---|---|
TableFunctionBuilder::with_state | a template S: Clone | a clone of the template |
TableFunctionBuilder::with_bind_init | immutable bind data B | init(&B) |
Use with_state when the scan state is cheap to clone. Use
with_bind_init when it is not (it holds a file handle, a large buffer, a
connection), or when the parameters and the cursor are naturally separate.
use quack_rs::prelude::*;
#[derive(Clone)]
struct State { remaining: u64 }
fn register(reg: &impl Registrar) -> ExtResult<()> {
let builder = TableFunctionBuilder::new("count_down")
.param(TypeId::BigInt)
.with_state::<State, _>(|bind| {
bind.add_result_column("n", TypeId::BigInt);
let raw = unsafe { bind.get_parameter_value(0) };
let n = raw.as_i64_or(0).max(0) as u64;
Ok(State { remaining: n })
})
.scan(|state, chunk| {
if state.remaining == 0 {
unsafe { chunk.set_size(0) };
return Ok(());
}
let mut writer = unsafe { chunk.writer(0) };
unsafe { writer.write_i64(0, state.remaining as i64) };
state.remaining -= 1;
unsafe { chunk.set_size(1) };
Ok(())
})
.build()?;
unsafe { reg.register_table(builder) }
}§Design
- The
bindclosure runs once per bind — once per query plan, not once per execution. It receives aBindInfo, declares the output schema, and reads parameters. DuckDBkeeps that one bind result for the lifetime of the plan and runsinitagainst it every time the plan executes: eachEXECUTEof a prepared statement, each iteration of a recursive CTE that references the function, and so on. The builder therefore never moves state out of the bind result; each execution gets a freshS(a clone of the template, or the result of yourinitclosure).- The
scanclosure runs repeatedly until it sets the output chunk size to zero. It receives&mut Sand aDataChunkfor output. - Panics in user closures are caught via
std::panic::catch_unwind; the error is reported throughDuckDBand the chunk size is forced to zero to safely terminate the scan.
§Threading
Because S is only required to be Send + 'static (not Sync), the typed
builder forces scans to execute on a single worker via
InitInfo::set_max_threads with
1. Extensions that want true multi-worker parallelism should continue to
use the raw TableFunctionBuilder API and split state across
local_init.
§No projection pushdown
The typed builder does not offer projection_pushdown,
build returns an error if it was
switched on on the raw builder before with_state / with_bind_init, and
registering the builder build returns fails if it was switched on
afterwards. So does replacing its bind, init, local_init, scan or
extra_info: the typed callbacks read one another’s data.
With pushdown on,
DuckDB hands the scan a chunk holding only the projected columns, in
projection order, so chunk.writer(0) is no longer “the first declared
column” — and the scan closure has no way to learn the mapping. A scan
written against the declared schema would then write column a’s values
into column b. Use the raw TableFunctionBuilder with
InitInfo::projected_column_index
when you need pushdown.
use quack_rs::prelude::*;
// Does not compile: the typed builder has no `projection_pushdown`.
let _ = TableFunctionBuilder::new("two_cols")
.with_state(|_bind| Ok(0_u8))
.projection_pushdown(true);Structs§
- Typed
Table Function Builder - Closure-based builder for table functions with a typed, mutable scan state.