Skip to main content

FileHandle

Struct FileHandle 

Source
pub struct FileHandle<'fs> { /* private fields */ }
Expand description

RAII wrapper for an open duckdb_file_handle.

Automatically closed and destroyed when dropped.

§Lifetime

A handle refers to the database’s underlying file system: once the database is closed (duckdb_close on the last reference), every method is a use-after-free — valgrind reports an invalid read in duckdb::FileHandle::Read. The handle therefore borrows the FileSystem that opened it for 'fs, which in turn borrows the ClientContext whose contract keeps the connection (and so the database) open. Using a handle after its file system is gone does not compile:

ⓘ
use quack_rs::client_context::ClientContext;
use quack_rs::file_system::{FileOpenOptions, FileSystem};

fn demo(ctx: &ClientContext) {
    let fs = FileSystem::from_client_context(ctx).unwrap();
    let handle = fs.open(c"data.csv", &FileOpenOptions::read_only()).unwrap();
    drop(fs); // error[E0505]: cannot move out of `fs` because it is borrowed
    let _ = handle.size();
}

§Threads

A FileHandle is Send: it may be moved to, used on and dropped on another thread. It is not Sync, so two threads cannot use one at the same time; share it behind a Mutex. See the unsafe impl Send below for the reasoning.

ⓘ
fn assert_sync<T: Sync>() {}
assert_sync::<quack_rs::file_system::FileHandle<'static>>();

§Keeping a file open across callbacks

A COPY ... TO writer opens its file in global_init, writes it in sink and closes it in finalize, so the handle has to live in the statement’s global state, which is 'static and shared by every sink thread. The borrow above cannot express that the state is destroyed while the statement — and so the connection and database — are still alive, so end it explicitly, once, and keep the handle in a Mutex:

use std::sync::Mutex;
use quack_rs::copy_function::CopyGlobalInitInfo;
use quack_rs::file_system::{FileHandle, FileOpenOptions, FileSystem};

struct Writer {
    file: Mutex<FileHandle<'static>>,
}

// SAFETY: the context is only used inside this callback.
let ctx = unsafe { info.get_client_context() };
let fs = FileSystem::from_client_context(&ctx).ok_or("no file system")?;
// SAFETY: DuckDB copies the path out before the callback returns.
let path = unsafe { info.get_file_path() }.map_err(|e| e.to_string())?;
let path = std::ffi::CString::new(path).map_err(|e| e.to_string())?;
let file = fs.open(&path, &FileOpenOptions::write_create()).map_err(|e| e.to_string())?;
// SAFETY: the handle ends up in this statement's global state, which
// DuckDB destroys before the statement finishes, while the connection and
// the database it was opened against are still open.
let file: FileHandle<'static> = unsafe { FileHandle::from_raw(file.into_raw()) };
info.set_global_state_typed(Writer { file: Mutex::new(file) });

sink then reads it with global_state_ref::<Writer>() and locks the mutex to write; dropping the state closes the file.

Implementations§

Source§

impl FileHandle<'_>

Source

pub const unsafe fn from_raw(handle: duckdb_file_handle) -> Self

Wraps a raw duckdb_file_handle, taking ownership.

§Safety

handle must be a valid, non-null duckdb_file_handle that the caller no longer manages, and the database it was opened against must stay open for the whole of the lifetime the caller picks for the result.

Source

pub const fn as_raw(&self) -> duckdb_file_handle

Returns the raw handle.

Source

pub const fn into_raw(self) -> duckdb_file_handle

Consumes the FileHandle and returns the raw handle without closing it, the inverse of from_raw.

The caller takes over closing (duckdb_file_handle_close) and destroying (duckdb_destroy_file_handle) it — or hands it back to from_raw, which is how a handle is moved into state that outlives this borrow (see Keeping a file open across callbacks).

Source

pub fn read(&self, buf: &mut [u8]) -> Result<usize, ErrorData>

Reads up to buf.len() bytes into buf, returning the number of bytes read (0 at end of file).

§Errors

Returns the structured ErrorData on read failure.

Source

pub fn write(&self, buf: &[u8]) -> Result<usize, ErrorData>

Writes up to buf.len() bytes from buf, returning the number written.

§Errors

Returns the structured ErrorData on write failure.

Source

pub fn read_exact(&self, buf: &mut [u8]) -> Result<(), ErrorData>

Reads exactly buf.len() bytes, or fails.

duckdb_file_handle_read returns “the number of bytes actually read”, so a single read can come up short on any file system — and httpfs is exactly where that happens. This loops until the buffer is full.

§Errors

Returns the structured ErrorData on read failure, or a DuckDbErrorType::Io error if the file ends before buf is filled.

Source

pub fn read_to_end(&self, buf: &mut Vec<u8>) -> Result<usize, ErrorData>

Appends the rest of the file to buf, returning how many bytes were added.

Reads from the current position to end of file, looping over short reads.

§Errors

Returns the structured ErrorData on read failure.

Source

pub fn write_all(&self, buf: &[u8]) -> Result<(), ErrorData>

Writes all of buf, or fails.

Like read_exact, this exists because duckdb_file_handle_write reports the number of bytes actually written.

§Errors

Returns the structured ErrorData on write failure, or a DuckDbErrorType::Io error if DuckDB stops accepting bytes before the buffer is drained.

Source

pub fn seek(&self, position: u64) -> Result<(), ErrorData>

Seeks to an absolute byte position.

§Errors

Returns the structured ErrorData if the seek fails, and an InvalidInput error, without calling DuckDB, for a position past i64::MAX, which DuckDB’s seek cannot take (it used to be clamped, and a file system that accepts that offset returned Ok).

Source

pub fn tell(&self) -> Result<u64, ErrorData>

Returns the current byte offset within the file.

§Errors

Returns the structured ErrorData if DuckDB cannot report the position. The C API signals that with a negative return value, which is far too easy to clamp to zero by accident.

Source

pub fn size(&self) -> Result<u64, ErrorData>

Returns the total size of the file in bytes.

§Errors

Returns the structured ErrorData if DuckDB cannot report the size.

Source

pub fn sync(&self) -> Result<(), ErrorData>

Flushes buffered writes to durable storage.

§Errors

Returns the structured ErrorData if the sync fails.

Source

pub fn close(&self) -> Result<(), ErrorData>

Closes the file. The handle is still destroyed on drop.

§Errors

Returns the structured ErrorData if the close fails. Dropping the handle then tries to close it once more, and leaks it if that fails too: destroying it would retry the close where an exception aborts the process.

Source

pub fn error_data(&self) -> ErrorData

Returns the structured error from the most recent failed operation.

Trait Implementations§

Source§

impl Debug for FileHandle<'_>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Drop for FileHandle<'_>

Source§

fn drop(&mut self)

Executes the destructor for this type. Read more
Source§

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)
Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned. Read more
Source§

impl Send for FileHandle<'_>

Auto Trait Implementations§

§

impl<'fs> !Sync for FileHandle<'fs>

§

impl<'fs> Freeze for FileHandle<'fs>

§

impl<'fs> RefUnwindSafe for FileHandle<'fs>

§

impl<'fs> Unpin for FileHandle<'fs>

§

impl<'fs> UnsafeUnpin for FileHandle<'fs>

§

impl<'fs> UnwindSafe for FileHandle<'fs>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.