Expand description
DuckDB C Extension API ABI compatibility checking.
§Why this module exists
A loadable extension does not link against DuckDB’s symbols. Instead,
DuckDB hands the extension a pointer to a duckdb_ext_api_v1 struct — an
array of function pointers — and the extension calls through it. The
extension’s compiled-in idea of that struct’s layout comes from the
duckdb_extension.h shipped with the libduckdb-sys version Cargo resolved
at build time. DuckDB’s idea of the layout comes from whatever DuckDB
binary is doing the loading. If those two layouts disagree, every call
lands on the wrong function pointer.
duckdb_ext_api_v1 is split into two regions:
| Region | Slots | Guarantee |
|---|---|---|
| Stable | 0 .. STABLE_API_SLOT_COUNT | Frozen since DuckDB v1.2.0: ABI-identical in every release from v1.2.0 through v2.0.0 (two slots were renamed varint → bignum in v1.4.0; the struct layout is unchanged). |
| Unstable | STABLE_API_SLOT_COUNT .. | Before v1.5.6, DuckDB twice inserted new entries in the middle, shifting every later slot. |
The unstable region is where duckdb-1-5 lives: scalar bind/init, copy
functions, catalog access, ErrorData, FileSystem, Expression,
SelectionVector, config options, table descriptions, TIME_NS values, and
the client context all sit past the stable boundary.
The unstable region grew at every minor version in the table below, and
twice DuckDB inserted entries into its middle, shifting every later
slot. The releases from v1.4.4 on were checked by hashing
extension_api.hpp at each tag (v1.4.4 and v1.4.5 are identical, as are
v1.5.0 and v1.5.1, and v1.5.2 to v1.5.5); the rest of the table comes from
the original survey. scripts/check-abi-table.py re-derives the whole table
from every release’s duckdb_extension.h, fingerprinting each layout by its
full declarations, and found v1.5.6 identical to v1.5.2 – v1.5.5, and
v2.0.0’s layout to be v1.5.6’s with two slots appended.
DuckDB | Total slots | What changed |
|---|---|---|
| v1.2.0 – v1.2.2 | 408 | baseline |
| v1.3.0 – v1.3.2 | 428 | appended |
| v1.4.0 – v1.4.5 | 459 | duckdb_create_varint → duckdb_create_bignum, appended |
| v1.5.0 – v1.5.1 | 545 | duckdb_appender_clear inserted at slot 410 |
| v1.5.2 – v1.5.6 | 546 | duckdb_geometry_type_get_crs inserted at slot 493 |
| v2.0.0 | 548 | duckdb_create_timestamp_tz_ns, duckdb_get_timestamp_tz_ns appended |
DuckDB v1.5.6 stabilised the whole struct: its header declares all 546
slots stable for an extension that targets C API v1.5.6, where earlier
releases declared only the first 357 stable. The layout is unchanged, and
so is what this module checks: quack-rs targets C API v1.2.0, and every
release before v1.5.6 still hands out the 546-slot struct with nothing past
slot 357 guaranteed. v2.0.0 keeps the V1 C API at version v1.5.6 and only
appends to it, so a build against the v1.5.2 – v1.5.6 layout runs on
v2.0.0 unchanged; check accepts it there (PREFIX_LAYOUTS).
§Why DuckDB does not catch this for you
DuckDB’s loader validates the extension footer’s ABI metadata:
C_STRUCT+ a C API version (v1.2.0) — accepted by anyDuckDBwhose C API version is greater than or equal to it, then handed the whole struct including the unstable region. No engine-version check at all.DuckDBv1.2.0 – v1.5.5 declare C API v1.2.0; v1.5.6 and v2.0.0 declare v1.5.6.C_STRUCT_UNSTABLE+ aDuckDBrelease version — accepted only by that exactDuckDB1.x release.DuckDB2.0 sends every such binary to the C API v2 entry point instead (see below).
An extension that touches the unstable region but is stamped C_STRUCT
therefore loads happily into a DuckDB with a different layout and then
mis-dispatches. For example, an extension built against DuckDB v1.5.0 and
loaded into v1.5.2+ would call duckdb_scalar_function_set_bind_data(info, data, destroy) and actually invoke
duckdb_scalar_function_get_client_context(info).
§What this module does
check compares the slot count of the layout this extension was compiled
against with the slot count of the layout the running DuckDB uses (looked
up from duckdb_library_version(), which lives at stable slot 7 and is
therefore always safe to call). A mismatch is reported as
AbiCheck::LayoutMismatch instead of being allowed to corrupt memory.
init_extension and
init_extension_v2 run this check
automatically under AbiPolicy::Strict, which is the default whenever the
duckdb-1-5 feature is enabled. Extensions that only use the stable region
are unaffected and stay portable across every DuckDB release.
§The bindings read the whole struct before this check runs
libduckdb-sys generates its duckdb_ext_api_v1 with
DUCKDB_EXTENSION_API_VERSION_UNSTABLE defined, so the Rust struct always
has every slot of the release it was generated from (546 for the v1.5.x
bindings, 548 for v2.0), whatever quack-rs features are on. Its
duckdb_rs_extension_api_init asks DuckDB for the struct (get_api) and
then reads every one of those slots through the returned pointer,
storing each non-null one in a process-wide table. The entry point must call
it before check, because check reaches duckdb_library_version
through that table.
DuckDB’s get_api returns the address of the duckdb_ext_api_v1 member
of the loader’s per-load state (DuckDBExtensionLoadState::api_struct in
src/main/extension/extension_load.cpp, a local of the loading call), sized
by the engine’s own header. So loading a build against the 546-slot
bindings into DuckDB v1.4.x (459 slots) reads 87 pointers, 696 bytes on a
64-bit target, past the end of that member, and into v1.5.0 – v1.5.1 (545
slots) one pointer: the fields after it in the load state, then the
loader’s stack. This happens on every such LOAD, before any check,
including for a build without duckdb-1-5 that check then lets load as
AbiCheck::StableOnly. The memory is mapped (no crash has been observed or
is expected), and the values it yields are stored but never called by code
that stays in the stable prefix, or when the check refuses the load. They
are what an unstable-region wrapper would call after
AbiPolicy::Warn or AbiPolicy::Trust let a mismatched layout load —
one more reason those policies are undefined behaviour on a mismatch. The
over-read is in libduckdb-sys’s generated code; quack-rs could avoid it
only by calling get_api and reading slot 7 itself before handing the
struct to the bindings.
§A DuckDB release the table has not seen
check does not extrapolate: a duckdb-1-5 build under
AbiPolicy::Strict refuses DuckDB v2.0.1 (or any release missing from the
table) as AbiCheck::UnknownEngineVersion until quack-rs lists it, or until
the build declares the engine with QUACK_RS_TARGET_DUCKDB_VERSION. A
default-feature build is AbiCheck::StableOnly and loads there. The layout
is not assumed to carry over to a 2.0.x patch release: DuckDB v2.0.0’s
duckdb_extension.h opens with
// WARNING: this API is not yet stable, meaning it is only guaranteed to work for this specific DuckDB version.,
v2.0.0 itself added two slots to the band v1.5.6 had declared stable, and a
1.5.x patch release (v1.5.2) once inserted a slot mid-struct.
§Shipping an extension that uses the unstable API
Build against DuckDB v1.5.2 or newer bindings and stamp the binary
--abi-type C_STRUCT --duckdb-version v1.5.6 with
append_metadata (with extension-ci-tools:
TARGET_DUCKDB_VERSION=v1.5.6 and no USE_UNSTABLE_C_API). DuckDB
v1.5.0 – v1.5.5 then refuse it at LOAD (their C API is v1.2.0), and
v1.5.6 and v2.0.0 load it: v1.5.6 declared that whole layout stable, and
v2.0.0 only appended to it. The runtime guard in this module is the
belt-and-braces for builds where that metadata is missing or wrong — which
includes every LOAD '/path/to/ext.duckdb_extension' during development.
Do not stamp such a build C_STRUCT_UNSTABLE any more.
DuckDB 2.0 sends every C_STRUCT_UNSTABLE binary to a
<name>_init_c_api_v2 entry point (the C API v2 in duckdb_extension_v2.h),
which quack-rs does not generate, so it fails to LOAD there with “did not
contain function <name>_init_c_api_v2”. On DuckDB 1.x it still pins the
binary to one release.
Enums§
- AbiCheck
- The result of an ABI layout check.
- AbiPolicy
- How
init_extensionshould react to the ABI layout check.
Constants§
- STABLE_
API_ SLOT_ COUNT - Number of function-pointer slots in the stable prefix of
duckdb_ext_api_v1.
Functions§
- built_
against_ version - The
DuckDBrelease this extension declares it was built against, if any. - check⚠
- Checks whether the running
DuckDBuses the sameduckdb_ext_api_v1layout this extension was compiled against. - compiled_
slot_ count - Number of
duckdb_ext_api_v1slots this extension was compiled against. - engine_
version ⚠ - Returns the version string reported by the running
DuckDB. - expected_
slot_ count - Returns the verified
duckdb_ext_api_v1slot count for aDuckDBrelease, orNoneif this build has no verified entry for it. - parse_
version - Parses a
DuckDBversion string into(major, minor, patch). - uses_
unstable_ api - Returns
trueif this build ofquack-rscan reach past the stable prefix.