Skip to main content

DataChunk

Struct DataChunk 

Source
pub struct DataChunk { /* private fields */ }
Expand description

A non-owning wrapper around a duckdb_data_chunk.

This wrapper does not destroy the chunk on drop — DuckDB owns the chunk and manages its lifetime. DataChunk simply provides ergonomic methods for accessing vectors and metadata within callback functions.

Implementations§

Source§

impl DataChunk

Source

pub const unsafe fn from_raw(raw: duckdb_data_chunk) -> Self

Wraps a raw duckdb_data_chunk handle.

§Safety

raw must be a valid duckdb_data_chunk obtained from a DuckDB callback (e.g., a scan callback’s output parameter or an aggregate update callback’s input chunk). The chunk must remain valid for the lifetime of this wrapper.

Source

pub fn size(&self) -> usize

Returns the number of rows in this data chunk.

Source

pub unsafe fn set_size(&self, size: usize)

Sets the number of rows in this data chunk.

Call this in scan callbacks after writing output rows. Set to 0 to signal end of stream.

§Safety

size must not exceed the chunk’s capacity (typically 2048).

Source

pub fn column_count(&self) -> usize

Returns the number of columns in this data chunk.

Source

pub unsafe fn any_null(&self, row: usize) -> bool

Returns true when any column of this chunk is NULL at row.

This is the SQL-standard NULL-propagation predicate for a scalar function: if any argument is NULL, the result is NULL.

§Why you have to ask

NullHandling::DefaultNullHandling does not make DuckDB propagate NULLs for a C API scalar function at run time — see propagate_nulls for the details and the evidence.

For a hot loop over many columns, prefer propagate_nulls, which resolves each column’s validity pointer once instead of once per row.

§Safety
  • row must be less than size.
  • The chunk must be a valid input chunk for the current callback.
Source

pub unsafe fn propagate_nulls(&self, output: &mut VectorWriter)

Marks output NULL at every row where any column of this chunk is NULL.

Each row is nulled through VectorWriter::set_null, so a STRUCT or ARRAY output has its fields / elements nulled at that row too, and (f(x)).a on a NULL row reads NULL rather than the value the callback wrote there.

§What DuckDB actually does with NULLs

It is natural to read NullHandling::DefaultNullHandling as “DuckDB returns NULL for me when an argument is NULL”. For a C API scalar function that is not what happens at run time:

  • CAPIScalarFunction calls the extension’s callback for the whole flattened chunk, NULL rows included, and never inspects the result’s validity.
  • ExpressionExecutor::Execute then calls VerifyNullHandling, which only asserts that the function already produced NULL wherever an input was NULL. Through DuckDB 1.5.6 it is compiled only under #ifdef DEBUG; in 2.0.0 it is compiled in but runs only after SET debug_verification_mode = 'VERIFY_FUNCTIONS', so a default session still believes the callback.

So in any release build of DuckDB — which is every build a user runs — a callback that ignores input validity returns a non-NULL answer for a NULL input, silently. SELECT f(NULL) still looks right, because a literal NULL is constant-folded before the function is ever called; the wrong answers only appear once the argument comes from a column. (Verified against DuckDB 1.5.4: a function returning 999 for every row returned 999, valid, for a NULL row of a real table.)

Call this after writing the output, and the SQL-standard behaviour is restored. Functions that mean to see NULLs — coalesce-likes, is_null-likes — should not call it, and should register with NullHandling::SpecialNullHandling so DuckDB’s debug assertion does not fire.

§Example
use quack_rs::data_chunk::DataChunk;
use quack_rs::vector::VectorWriter;

quack_rs::scalar_callback!(double_it, |_info, input, output| {
    let chunk = unsafe { DataChunk::from_raw(input) };
    let reader = unsafe { chunk.reader(0) };
    let mut writer = unsafe { VectorWriter::from_vector(output) };
    for row in 0..chunk.size() {
        unsafe { writer.write_i64(row, reader.read_i64(row) * 2) };
    }
    // Without this, `double_it(NULL)` is 0, not NULL.
    unsafe { chunk.propagate_nulls(&mut writer) };
});
§Safety
  • output must write to a vector with at least size rows — the output vector of the callback this chunk was passed to.
  • The chunk must be a valid input chunk for the current callback.
Source

pub unsafe fn vector(&self, col_idx: usize) -> duckdb_vector

Returns the raw duckdb_vector handle for the given column index.

§Safety

col_idx must be less than column_count.

Source

pub unsafe fn writer(&self, col_idx: usize) -> VectorWriter

Creates a VectorWriter for the given column index.

§Safety
  • col_idx must be less than column_count.
  • The chunk must be a writable output chunk (not a read-only input chunk).
  • The returned value borrows nothing: it must not be used after the underlying duckdb_data_chunk is destroyed — for an OwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Source

pub unsafe fn reader(&self, col_idx: usize) -> VectorReader

Creates a VectorReader for the given column index.

The reader’s row count is set to this chunk’s current size.

§Safety
  • col_idx must be less than column_count.
  • The returned value borrows nothing: it must not be used after the underlying duckdb_data_chunk is destroyed — for an OwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.

Build the reader once per chunk and read every row through it: this call makes several C API calls, which a chunk.reader(0).read_i64(row) inside the row loop pays on every row (measured at about 6× the cost of the read itself).

Source

pub unsafe fn struct_reader( &self, col_idx: usize, field_count: usize, ) -> StructReader

Creates a StructReader for a STRUCT column at the given index.

This is a convenience method that combines vector with StructReader::new.

§Safety
  • col_idx must be less than column_count.
  • The column at col_idx must have a STRUCT type with field_count fields.
  • The returned value borrows nothing: it must not be used after the underlying duckdb_data_chunk is destroyed — for an OwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Source

pub unsafe fn struct_field_reader( &self, col_idx: usize, field_idx: usize, ) -> VectorReader

Creates a VectorReader for a field of a STRUCT column.

Convenience for accessing a specific field in a STRUCT input column.

§Safety
  • col_idx must be less than column_count.
  • The column at col_idx must have a STRUCT type.
  • field_idx must be a valid field index within the STRUCT.
  • The returned value borrows nothing: it must not be used after the underlying duckdb_data_chunk is destroyed — for an OwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Source

pub unsafe fn struct_writer( &self, col_idx: usize, field_count: usize, ) -> StructWriter

Creates a StructWriter for a STRUCT column at the given index.

This is a convenience method that combines vector with StructWriter::new.

§Safety
  • col_idx must be less than column_count.
  • The column at col_idx must have a STRUCT type with field_count fields.
  • The chunk must be a writable output chunk.
  • The returned value borrows nothing: it must not be used after the underlying duckdb_data_chunk is destroyed — for an OwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Source

pub unsafe fn into_chunk_writer(self) -> ChunkWriter

Creates a ChunkWriter for this output data chunk.

The ChunkWriter counts the rows claimed with next_row() and not given back with release_row(), and calls set_size with that count on drop: every row claimed must be written or given back.

§Safety

This chunk must be a valid, writable output chunk from a table function scan callback.

Source

pub const fn as_raw(&self) -> duckdb_data_chunk

Returns the raw duckdb_data_chunk handle.

Trait Implementations§

Source§

impl Debug for DataChunk

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.