Expand description
Bulk data appending.
Appender is an RAII wrapper around DuckDB’s appender — the fastest way
to bulk-insert rows into an existing table, and considerably faster than
issuing INSERT statements.
§Two ways to append
Row at a time. Call one append_* per column, then
end_row — or let row call it for
you, which is the difference between a forgotten end_row being a compile
-time non-issue and a silently short table:
use quack_rs::appender::Appender;
// SAFETY: `con` is a valid, open connection.
let appender = unsafe { Appender::new(con, None, c"measurements") }?;
for (sensor, reading) in [("a", 1.5_f64), ("b", 2.5)] {
appender.row(|row| {
row.append_str(sensor)?;
row.append_f64(reading)
})?;
}
appender.close()?;A chunk at a time. Build a DataChunk and hand it over with
append_chunk. Fewer FFI crossings, and the
natural fit when the data already lives in vectors.
§Errors and the appender’s lifecycle
Appended rows are buffered. A constraint violation therefore surfaces at
flush or close, not at the
append_* call that caused it, and it invalidates every buffered row.
Dropping an Appender closes it, and a failure there has nowhere to go —
DuckDB’s own header is explicit that after destruction “it is no longer
possible to obtain the specific error message”. Call
close explicitly whenever the outcome matters.
§A row that fails half-way
DuckDB counts the values of the current row and has no way to take one
back: end_row wants every column, flush wants none, and its Close
flushes only in those two states — in any other it returns success and
writes nothing. A row whose closure fails after its first
value therefore leaves a half-written row that cannot be completed or
dropped, and with it every row buffered since the last flush is lost
(DuckDB also flushes on its own each time 204,800 rows accumulate; rows
before that are safe).
quack-rs tracks the row itself so that this is never silent:
- the appender becomes poisoned: every later append,
row,end_row,flushandclosereturns an error that says how many buffered rows were not written; closewith a row started but not ended (byrowor by hand) is an error rather than a silent no-op;- with
duckdb-1-5,cleardiscards the buffered rows and the half row and makes the appender usable again. Without it, a poisoned appender stays poisoned.
A value that fails as the first of its row loses nothing (DuckDB has
not counted it), and one that fails later in a row appended by hand can be
retried — only an abandoned row poisons. If losing buffered rows is not
acceptable, flush at the points you can afford to
lose work back to.
After a successful close the appender refuses further
work; DuckDB itself would accept appends and write them at the next
flush.
§A streaming result open on the same connection
A flush runs an INSERT on the appender’s connection, so it is “another
statement” to a streaming
QueryResult still open there. Up to
DuckDB 1.5.x the flush succeeds and the stream ends with an error. From
2.0.0 the flush — including the automatic one every 204,800 rows and the
one in close — is refused with a “Resource In Use” error, and because a
failed flush discards the buffer on 2.0.0, every buffered row is lost.
Finish or drop the stream first, or append through a connection of its
own.
§Schema changes while rows are buffered
Buffered rows are written by column position when they are flushed. If
another connection drops a column and adds one in between, a buffered
value lands in the new column (cast to its type) with no error. Flush
before a concurrent ALTER TABLE if that matters.
§Feature flags
The appender is available without any feature flag: DuckDB has kept
duckdb_appender_* in the frozen stable prefix of the extension API
(slots 281–291 and 330–356) since v1.2.0, so using it does not push an
extension onto the version-pinned unstable ABI. Three methods are the
exception and are gated on duckdb-1-5: error_data, clear and
append_default_to_chunk. They are not linked here because the links would
not resolve when the feature is off, which is exactly when a reader most
wants to know they exist.
That gate also picks the error type — see AppendError.
Structs§
- Appender
- RAII wrapper for a
duckdb_appender.
Type Aliases§
- Append
Error - The error type every fallible
Appenderoperation reports.