ferro-core

Core storage, metadata, search, and runtime abstractions for the Ferro platform. Provides production-ready storage backends (SQLite, PostgreSQL, S3, GCS, Azure Blob), a Tantivy-based full-text search engine, and a WASM worker runtime powered by Wasmtime.

Key Types

TypeDescription
InMemoryStorageEngineIn-memory StorageEngine for testing and development
ObjectStoreStorageEngineWraps any object_store implementation (S3, GCS, Azure)
SqliteMetadataStoreSQLite-backed metadata store
PgMetadataStorePostgreSQL-backed metadata store
InMemoryMetadataStoreIn-memory metadata store for testing
MetadataStoreTrait for pluggable metadata backends
SearchEngineTantivy full-text search with path, name, content, owner fields
WasmWorkerRuntimeSandboxed WASM execution with fuel limits and WASI support
SqlitePersistenceSnapshot and audit log persistence

Feature Flags

FeatureDefaultDescription
sqliteyesSQLite metadata store via sqlx
searchyesTantivy full-text search engine
wasmyesWasmtime WASM worker runtime
object_storeyesobject_store backend (local, S3, GCS, Azure)
s3noAWS S3 object store backend
gcsnoGoogle Cloud Storage backend
azurenoAzure Blob Storage backend
postgresnoPostgreSQL metadata store via sqlx

Minimal Usage

In-memory storage

#![allow(unused)]
fn main() {
use ferro_core::storage::InMemoryStorageEngine;
use ferro_common::storage::StorageEngine;
use bytes::Bytes;

let engine = InMemoryStorageEngine::new();
let meta = engine.put("/hello.txt", Bytes::from("world"), "user1").await?;
}

SQLite metadata store

#![allow(unused)]
fn main() {
use ferro_core::sqlx_metadata::SqliteMetadataStore;

let store = SqliteMetadataStore::new("sqlite:metadata.db").await?;
}
#![allow(unused)]
fn main() {
use ferro_core::search::SearchEngine;
use std::path::Path;

let engine = SearchEngine::new(Path::new("/tmp/ferro-index"))?;
engine.index("/docs/readme.txt", "Introduction to Ferro", &meta).await?;
let results = engine.search("introduction", 10).await?;
}

WASM worker

#![allow(unused)]
fn main() {
use ferro_core::wasm::{WasmWorkerRuntime, WorkerConfig};

let runtime = WasmWorkerRuntime::new(WorkerConfig::default())?;
runtime.register("*.md", &wasm_bytes, "process")?;
let result = runtime.execute("/docs/readme.md", &content).await?;
}

Object store with S3

# Cargo.toml
ferro-core = { version = "0.1", features = ["s3"] }
#![allow(unused)]
fn main() {
use ferro_core::ObjectStoreStorageEngine;
use object_store::aws::AmazonS3Builder;
use std::sync::Arc;

let s3 = AmazonS3Builder::new()
    .with_bucket_name("my-bucket")
    .build()?;
let engine = ObjectStoreStorageEngine::with_prefix(Arc::new(s3), "ferro");
}