Skip to main content

Module typed

Module typed 

Source
Expand description

Closure-based table functions with typed scan state.

This module provides TypedTableFunctionBuilder, a higher-level builder layered on top of TableFunctionBuilder. It lets extension authors write table functions using safe Rust closures instead of unsafe extern "C" fn trampolines for bind, init, and scan.

§Motivation

The raw TableFunctionBuilder API requires authors to write three hand-rolled unsafe extern "C" fn callbacks and manually shuttle state through FfiBindData / FfiInitData. For extensions that merely need “take some parameters at bind, stream rows until exhausted”, that ceremony is largely accidental complexity. TypedTableFunctionBuilder collapses it to closures.

§Two entry points

ConstructorBind producesEach execution starts from
TableFunctionBuilder::with_statea template S: Clonea clone of the template
TableFunctionBuilder::with_bind_initimmutable bind data Binit(&B)

Use with_state when the scan state is cheap to clone. Use with_bind_init when it is not (it holds a file handle, a large buffer, a connection), or when the parameters and the cursor are naturally separate.

use quack_rs::prelude::*;

#[derive(Clone)]
struct State { remaining: u64 }

fn register(reg: &impl Registrar) -> ExtResult<()> {
    let builder = TableFunctionBuilder::new("count_down")
        .param(TypeId::BigInt)
        .with_state::<State, _>(|bind| {
            bind.add_result_column("n", TypeId::BigInt);
            let raw = unsafe { bind.get_parameter_value(0) };
            let n = raw.as_i64_or(0).max(0) as u64;
            Ok(State { remaining: n })
        })
        .scan(|state, chunk| {
            if state.remaining == 0 {
                unsafe { chunk.set_size(0) };
                return Ok(());
            }
            let mut writer = unsafe { chunk.writer(0) };
            unsafe { writer.write_i64(0, state.remaining as i64) };
            state.remaining -= 1;
            unsafe { chunk.set_size(1) };
            Ok(())
        })
        .build()?;
    unsafe { reg.register_table(builder) }
}

§Design

  • The bind closure runs once per bind — once per query plan, not once per execution. It receives a BindInfo, declares the output schema, and reads parameters.
  • DuckDB keeps that one bind result for the lifetime of the plan and runs init against it every time the plan executes: each EXECUTE of a prepared statement, each iteration of a recursive CTE that references the function, and so on. The builder therefore never moves state out of the bind result; each execution gets a fresh S (a clone of the template, or the result of your init closure).
  • The scan closure runs repeatedly until it sets the output chunk size to zero. It receives &mut S and a DataChunk for output.
  • Panics in user closures are caught via std::panic::catch_unwind; the error is reported through DuckDB and the chunk size is forced to zero to safely terminate the scan.

§Threading

Because S is only required to be Send + 'static (not Sync), the typed builder forces scans to execute on a single worker via InitInfo::set_max_threads with 1. Extensions that want true multi-worker parallelism should continue to use the raw TableFunctionBuilder API and split state across local_init.

§No projection pushdown

The typed builder does not offer projection_pushdown, build returns an error if it was switched on on the raw builder before with_state / with_bind_init, and registering the builder build returns fails if it was switched on afterwards. So does replacing its bind, init, local_init, scan or extra_info: the typed callbacks read one another’s data. With pushdown on, DuckDB hands the scan a chunk holding only the projected columns, in projection order, so chunk.writer(0) is no longer “the first declared column” — and the scan closure has no way to learn the mapping. A scan written against the declared schema would then write column a’s values into column b. Use the raw TableFunctionBuilder with InitInfo::projected_column_index when you need pushdown.

ⓘ
use quack_rs::prelude::*;

// Does not compile: the typed builder has no `projection_pushdown`.
let _ = TableFunctionBuilder::new("two_cols")
    .with_state(|_bind| Ok(0_u8))
    .projection_pushdown(true);

Structs§

TypedTableFunctionBuilder
Closure-based builder for table functions with a typed, mutable scan state.