pub struct OwnedDataChunk { /* private fields */ }Expand description
A duckdb_data_chunk owned by this crate.
QueryResult::next_chunk hands out chunks that the caller owns and must
destroy. This type does that on drop and derefs to the borrowing
DataChunk wrapper, so all the usual readers apply.
Implementations§
Source§impl OwnedDataChunk
impl OwnedDataChunk
Sourcepub fn new(types: &[LogicalType]) -> Result<Self, ExtensionError>
pub fn new(types: &[LogicalType]) -> Result<Self, ExtensionError>
Creates an empty chunk with one column per entry of types.
The chunk holds zero rows and has room for capacity
of them. Fill its columns through writer and
friends, then set_size to the number of rows
written — for example to build the chunk that
Appender::append_chunk
takes, without dropping to duckdb_create_data_chunk.
DuckDB copies the types, so types may be dropped as soon as this
returns.
§Errors
Returns an error, without calling DuckDB, if types is empty: a chunk
with no columns trips a debug assertion in DuckDB’s
DataChunk::Initialize. Returns an error if DuckDB refuses to create
the chunk, which it does when a type is or contains ANY or INVALID,
and when allocating the chunk throws.
§Example
use quack_rs::query::OwnedDataChunk;
use quack_rs::types::{LogicalType, TypeId};
let chunk = OwnedDataChunk::new(&[LogicalType::new(TypeId::BigInt)])?;
// SAFETY: column 0 is BIGINT and row 0 is within the chunk's capacity.
unsafe {
chunk.writer(0).write_i64(0, 42);
chunk.set_size(1);
}Sourcepub fn capacity(&self) -> usize
pub fn capacity(&self) -> usize
The number of rows a chunk made by new has room for:
duckdb_vector_size(), which is 2048 unless DuckDB was built with a
different STANDARD_VECTOR_SIZE.
A chunk handed out by QueryResult::next_chunk
has the same capacity.
Sourcepub fn reset(&mut self)
pub fn reset(&mut self)
Empties the chunk so it can be filled again: its size becomes zero and its columns are reset to fresh, writable vectors.
Readers and writers taken from the chunk before the reset must not be used after it, since the vectors they point into may have been replaced.
Methods from Deref<Target = DataChunk>§
Sourcepub unsafe fn set_size(&self, size: usize)
pub unsafe fn set_size(&self, size: usize)
Sets the number of rows in this data chunk.
Call this in scan callbacks after writing output rows. Set to 0 to
signal end of stream.
§Safety
size must not exceed the chunk’s capacity (typically 2048).
Sourcepub fn column_count(&self) -> usize
pub fn column_count(&self) -> usize
Returns the number of columns in this data chunk.
Sourcepub unsafe fn any_null(&self, row: usize) -> bool
pub unsafe fn any_null(&self, row: usize) -> bool
Returns true when any column of this chunk is NULL at row.
This is the SQL-standard NULL-propagation predicate for a scalar function: if any argument is NULL, the result is NULL.
§Why you have to ask
NullHandling::DefaultNullHandling
does not make DuckDB propagate NULLs for a C API scalar function at
run time — see
propagate_nulls for the details and the
evidence.
For a hot loop over many columns, prefer
propagate_nulls, which resolves each column’s
validity pointer once instead of once per row.
§Safety
rowmust be less thansize.- The chunk must be a valid input chunk for the current callback.
Sourcepub unsafe fn propagate_nulls(&self, output: &mut VectorWriter)
pub unsafe fn propagate_nulls(&self, output: &mut VectorWriter)
Marks output NULL at every row where any column of this chunk is NULL.
Each row is nulled through VectorWriter::set_null, so a STRUCT or
ARRAY output has its fields / elements nulled at that row too, and
(f(x)).a on a NULL row reads NULL rather than the value the callback
wrote there.
§What DuckDB actually does with NULLs
It is natural to read
NullHandling::DefaultNullHandling
as “DuckDB returns NULL for me when an argument is NULL”. For a C API
scalar function that is not what happens at run time:
CAPIScalarFunctioncalls the extension’s callback for the whole flattened chunk, NULL rows included, and never inspects the result’s validity.ExpressionExecutor::Executethen callsVerifyNullHandling, which only asserts that the function already produced NULL wherever an input was NULL. ThroughDuckDB1.5.6 it is compiled only under#ifdef DEBUG; in 2.0.0 it is compiled in but runs only afterSET debug_verification_mode = 'VERIFY_FUNCTIONS', so a default session still believes the callback.
So in any release build of DuckDB — which is every build a user runs —
a callback that ignores input validity returns a non-NULL answer for a
NULL input, silently. SELECT f(NULL) still looks right, because a
literal NULL is constant-folded before the function is ever called;
the wrong answers only appear once the argument comes from a column.
(Verified against DuckDB 1.5.4: a function returning 999 for every
row returned 999, valid, for a NULL row of a real table.)
Call this after writing the output, and the SQL-standard behaviour is
restored. Functions that mean to see NULLs — coalesce-likes,
is_null-likes — should not call it, and should register with
NullHandling::SpecialNullHandling
so DuckDB’s debug assertion does not fire.
§Example
use quack_rs::data_chunk::DataChunk;
use quack_rs::vector::VectorWriter;
quack_rs::scalar_callback!(double_it, |_info, input, output| {
let chunk = unsafe { DataChunk::from_raw(input) };
let reader = unsafe { chunk.reader(0) };
let mut writer = unsafe { VectorWriter::from_vector(output) };
for row in 0..chunk.size() {
unsafe { writer.write_i64(row, reader.read_i64(row) * 2) };
}
// Without this, `double_it(NULL)` is 0, not NULL.
unsafe { chunk.propagate_nulls(&mut writer) };
});§Safety
outputmust write to a vector with at leastsizerows — the output vector of the callback this chunk was passed to.- The chunk must be a valid input chunk for the current callback.
Sourcepub unsafe fn vector(&self, col_idx: usize) -> duckdb_vector
pub unsafe fn vector(&self, col_idx: usize) -> duckdb_vector
Returns the raw duckdb_vector handle for the given column index.
§Safety
col_idx must be less than column_count.
Sourcepub unsafe fn writer(&self, col_idx: usize) -> VectorWriter
pub unsafe fn writer(&self, col_idx: usize) -> VectorWriter
Creates a VectorWriter for the given column index.
§Safety
col_idxmust be less thancolumn_count.- The chunk must be a writable output chunk (not a read-only input chunk).
- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn reader(&self, col_idx: usize) -> VectorReader
pub unsafe fn reader(&self, col_idx: usize) -> VectorReader
Creates a VectorReader for the given column index.
The reader’s row count is set to this chunk’s current size.
§Safety
col_idxmust be less thancolumn_count.- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Build the reader once per chunk and read every row through it: this
call makes several C API calls, which a chunk.reader(0).read_i64(row)
inside the row loop pays on every row (measured at about 6× the cost of
the read itself).
Sourcepub unsafe fn struct_reader(
&self,
col_idx: usize,
field_count: usize,
) -> StructReader
pub unsafe fn struct_reader( &self, col_idx: usize, field_count: usize, ) -> StructReader
Creates a StructReader for a STRUCT column at the given index.
This is a convenience method that combines vector with
StructReader::new.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type withfield_countfields. - The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn struct_field_reader(
&self,
col_idx: usize,
field_idx: usize,
) -> VectorReader
pub unsafe fn struct_field_reader( &self, col_idx: usize, field_idx: usize, ) -> VectorReader
Creates a VectorReader for a field of a STRUCT column.
Convenience for accessing a specific field in a STRUCT input column.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type. field_idxmust be a valid field index within the STRUCT.- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.
Sourcepub unsafe fn struct_writer(
&self,
col_idx: usize,
field_count: usize,
) -> StructWriter
pub unsafe fn struct_writer( &self, col_idx: usize, field_count: usize, ) -> StructWriter
Creates a StructWriter for a STRUCT column at the given index.
This is a convenience method that combines vector with
StructWriter::new.
§Safety
col_idxmust be less thancolumn_count.- The column at
col_idxmust have a STRUCT type withfield_countfields. - The chunk must be a writable output chunk.
- The returned value borrows nothing: it must not be used after the
underlying
duckdb_data_chunkis destroyed — for anOwnedDataChunk, after it is dropped; in a callback, after the callback returns. It caches pointers into the chunk’s buffers, so a read afterwards is a use-after-free.