pub struct DataChunk { /* private fields */ }Expand description
A non-owning wrapper around a duckdb_data_chunk.
This wrapper does not destroy the chunk on drop — DuckDB owns the
chunk and manages its lifetime. DataChunk simply provides ergonomic
methods for accessing vectors and metadata within callback functions.
Implementations§
Source§impl DataChunk
impl DataChunk
Sourcepub const unsafe fn from_raw(raw: duckdb_data_chunk) -> Self
pub const unsafe fn from_raw(raw: duckdb_data_chunk) -> Self
Wraps a raw duckdb_data_chunk handle.
§Safety
raw must be a valid duckdb_data_chunk obtained from a DuckDB
callback (e.g., a scan callback’s output parameter or an aggregate
update callback’s input chunk). The chunk must remain valid for
the lifetime of this wrapper.
Sourcepub unsafe fn set_size(&self, size: usize)
pub unsafe fn set_size(&self, size: usize)
Sets the number of rows in this data chunk.
Call this in scan callbacks after writing output rows. Set to 0 to
signal end of stream.
§Safety
size must not exceed the chunk’s capacity (typically 2048).
Sourcepub fn column_count(&self) -> usize
pub fn column_count(&self) -> usize
Returns the number of columns in this data chunk.
Sourcepub unsafe fn any_null(&self, row: usize) -> bool
pub unsafe fn any_null(&self, row: usize) -> bool
Returns true when any column of this chunk is NULL at row.
This is the SQL-standard NULL-propagation predicate for a scalar function: if any argument is NULL, the result is NULL.
§Why you have to ask
NullHandling::DefaultNullHandling
does not make DuckDB propagate NULLs for a C API scalar function at
run time — see
propagate_nulls for the details and the
evidence.
For a hot loop over many columns, prefer
propagate_nulls, which resolves each column’s
validity pointer once instead of once per row.
§Safety
rowmust be less thansize.- The chunk must be a valid input chunk for the current callback.
Sourcepub unsafe fn propagate_nulls(&self, output: &mut VectorWriter)
pub unsafe fn propagate_nulls(&self, output: &mut VectorWriter)
Marks output NULL at every row where any column of this chunk is NULL.
Each row is nulled through VectorWriter::set_null, so a STRUCT or
ARRAY output has its fields / elements nulled at that row too, and
(f(x)).a on a NULL row reads NULL rather than the value the callback
wrote there.
§What DuckDB actually does with NULLs
It is natural to read
NullHandling::DefaultNullHandling
as “DuckDB returns NULL for me when an argument is NULL”. For a C API
scalar function that is not what happens at run time:
CAPIScalarFunctioncalls the extension’s callback for the whole flattened chunk, NULL rows included, and never inspects the result’s validity.ExpressionExecutor::Executethen callsVerifyNullHandling, which only asserts that the function already produced NULL wherever an input was NULL. ThroughDuckDB1.5.6 it is compiled only under#ifdef DEBUG; in 2.0.0 it is compiled in but runs only afterSET debug_verification_mode = 'VERIFY_FUNCTIONS', so a default session still believes the callback.
So in any release build of DuckDB — which is every build a user runs —
a callback that ignores input validity returns a non-NULL answer for a
NULL input, silently. SELECT f(NULL) still looks right, because a
literal NULL is constant-folded before the function is ever called;
the wrong answers only appear once the argument comes from a column.
(Verified against DuckDB 1.5.4: a function returning 999 for every
row returned 999, valid, for a NULL row of a real table.)
Call this after writing the output, and the SQL-standard behaviour is
restored. Functions that mean to see NULLs — coalesce-likes,
is_null-likes — should not call it, and should register with
NullHandling::SpecialNullHandling
so DuckDB’s debug assertion does not fire.
§Example
use quack_rs::data_chunk::DataChunk;
use quack_rs::vector::VectorWriter;
quack_rs::scalar_callback!(double_it, |_info, input, output| {
let chunk = unsafe { DataChunk::from_raw(input) };
let reader = unsafe { chunk.reader(0) };
let mut writer = unsafe { VectorWriter::from_vector(output) };
for row in 0..chunk.size() {
unsafe { writer.write_i64(row, reader.read_i64(row) * 2) };
}
// Without this, `double_it(NULL)` is 0, not NULL.
unsafe { chunk.propagate_nulls(&mut writer) };
});§Safety
outputmust write to a vector with at leastsizerows — the output vector of the callback this chunk was passed to.- The chunk must be a valid input chunk for the current callback.
Sourcepub unsafe fn vector(&self, col_idx: usize) -> duckdb_vector
pub unsafe fn vector(&self, col_idx: usize) -> duckdb_vector
Returns the raw duckdb_vector handle for the given column index.
§Safety
col_idx must be less than column_count.
Sourcepub unsafe fn writer(&self, col_idx: usize) -> VectorWriter
pub unsafe fn writer(&self, col_idx: usize) -> VectorWriter
Creates a VectorWriter for the given column index.
§Safety
col_idxmust be less thancolumn_count.- The chunk must be a writable output chunk (not a read-only input chunk).
- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn reader(&self, col_idx: usize) -> VectorReader
pub unsafe fn reader(&self, col_idx: usize) -> VectorReader
Creates a VectorReader for the given column index.
The reader’s row count is set to this chunk’s current size.
§Safety
col_idxmust be less thancolumn_count.- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Build the reader once per chunk and read every row through it: this
call makes several C API calls, which a chunk.reader(0).read_i64(row)
inside the row loop pays on every row (measured at about 6× the cost of
the read itself).
Sourcepub unsafe fn struct_reader(
&self,
col_idx: usize,
field_count: usize,
) -> StructReader
pub unsafe fn struct_reader( &self, col_idx: usize, field_count: usize, ) -> StructReader
Creates a StructReader for a STRUCT column at the given index.
This is a convenience method that combines vector with
StructReader::new.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type withfield_countfields. - The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn struct_field_reader(
&self,
col_idx: usize,
field_idx: usize,
) -> VectorReader
pub unsafe fn struct_field_reader( &self, col_idx: usize, field_idx: usize, ) -> VectorReader
Creates a VectorReader for a field of a STRUCT column.
Convenience for accessing a specific field in a STRUCT input column.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type. field_idxmust be a valid field index within the STRUCT.- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn struct_writer(
&self,
col_idx: usize,
field_count: usize,
) -> StructWriter
pub unsafe fn struct_writer( &self, col_idx: usize, field_count: usize, ) -> StructWriter
Creates a StructWriter for a STRUCT column at the given index.
This is a convenience method that combines vector with
StructWriter::new.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type withfield_countfields. - The chunk must be a writable output chunk.
- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn into_chunk_writer(self) -> ChunkWriter
pub unsafe fn into_chunk_writer(self) -> ChunkWriter
Creates a ChunkWriter for this output data chunk.
The ChunkWriter counts the rows claimed with
next_row() and not given back with
release_row(), and calls set_size with
that count on drop: every row claimed must be written or given back.
§Safety
This chunk must be a valid, writable output chunk from a table function scan callback.