Expand description
Safe construction of LIST and MAP output vectors.
§The hazard this exists to remove
A LIST vector stores a { offset, length } entry per row, plus one flat
child vector holding every row’s elements end to end. Writing one means:
duckdb_list_vector_reserve— make room in the child,- write the elements into the child,
duckdb_list_vector_set_size— declare how many are valid,- write each parent row’s
{ offset, length }entry.
Step 1 is the trap. duckdb_list_vector_reserve takes a total capacity,
not an increment, and when it grows it calls Vector::Resize, which
reallocates the child’s data buffer. Any VectorWriter made from the
child before that call now holds a dangling pointer — and the natural way to
write a list row by row is to keep one writer and reserve as you go:
// WRONG: `writer` dangles after the second reserve grows the child.
let mut writer = unsafe { ListVector::child_writer(vec) };
for (row, items) in rows.iter().enumerate() {
unsafe { ListVector::reserve(vec, total + items.len()) }; // may realloc
for (i, item) in items.iter().enumerate() {
unsafe { writer.write_i64(total + i, *item) }; // use-after-free
}
total += items.len();
}ListBuilder keeps one child writer, drops it after every reserve that
grows the child (and fetches a fresh one on the next row), tracks the running
offset, and writes the parent entries for you, so the sequence cannot be got
wrong.
§Example
use quack_rs::vector::ListBuilder;
let rows = [vec![1_i64, 2, 3], vec![], vec![42]];
// SAFETY: `list_vector` is a LIST output vector with BIGINT elements.
let mut builder = unsafe { ListBuilder::new(list_vector) };
for (row, items) in rows.iter().enumerate() {
// SAFETY: the child vector holds BIGINT, and `row` is in bounds.
unsafe {
builder.push_row(row, items.len(), |writer, base| {
for (i, item) in items.iter().enumerate() {
writer.write_i64(base + i, *item);
}
});
}
}
// SAFETY: every element promised by `push_row` was written.
unsafe { builder.finish() };§MAP
A MAP is a LIST<STRUCT{key, value}>, so the same builder drives it —
push_map_row hands the closure a writer for
the key child and one for the value child.
Structs§
- List
Builder - Incremental builder for a
LIST(orMAP) output vector.
Constants§
- MAX_
LIST_ CHILD_ CAPACITY DuckDB’s ceiling on one buffer of a child vector, in bytes (DConstants::MAX_VECTOR_SIZE).
Functions§
- max_
child_ ⚠capacity - The most child elements a
LISTorMAPvector’s child can be reserved for withoutDuckDBthrowing through the C API.