【文章标题】:TurboKV: Insanely fast Rust key-value store 【文章标题】:TurboKV:极速Rust键值存储
【文章正文】: TurboKV is an async embedded key-value database with atomic batches, ordered range scans, configurable durability, compression, and background compaction. TurboKV是一个异步嵌入式键值数据库,支持原子批处理、有序范围扫描、可配置持久性、压缩和后台压缩。
cargo add turbokv cargo add tokio —features full Or add the dependencies directly: [dependencies] turbokv = “0.6” tokio = { version = “1”, features = [“full”] } 运行 cargo add turbokv 运行 cargo add tokio —features full 或直接添加依赖项: [dependencies] turbokv = “0.6” tokio = { version = “1”, features = [“full”] }
TurboKV’s persisted Bloom-filter format uses hardware AES. Build x86/x86_64 targets with RUSTFLAGS=“-C target-feature=+aes,+sse2”, and ARM/AArch64 targets with RUSTFLAGS=“-C target-feature=+aes,+neon”. You may instead use -C target-cpu=native when the binary will run only on the same CPU model or a feature superset. TurboKV的持久化布隆过滤器格式使用硬件AES指令。构建x86/x86_64目标时使用RUSTFLAGS=“-C target-feature=+aes,+sse2”,ARM/AArch64目标使用RUSTFLAGS=“-C target-feature=+aes,+neon”。当二进制文件仅在相同CPU型号或功能超集上运行时,可使用-C target-cpu=native。
use turbokv::{Db, DbOptions, WriteBatch}; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let db = Db::open_with_options(”./my-database”, DbOptions::durable()).await?; db.insert(b”user:1”, b”Ada”).await?; assert_eq!(db.get(b”user:1”).await?, Some(b”Ada”.to_vec())); let mut batch = WriteBatch::new(); batch.put(b”user:2”, b”Grace”); batch.put(b”user:3”, b”Linus”); batch.delete(b”user:1”); db.write_batch(&batch).await?; for (key, value) in db.scan_prefix(b”user:“).await? { println!( ”{} = {}”, String::from_utf8_lossy(&key), String::from_utf8_lossy(&value) ); } db.close().await?; Ok(()) } 使用示例: use turbokv::{Db, DbOptions, WriteBatch}; #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let db = Db::open_with_options(”./my-database”, DbOptions::durable()).await?; db.insert(b”user:1”, b”Ada”).await?; assert_eq!(db.get(b”user:1”).await?, Some(b”Ada”.to_vec())); let mut batch = WriteBatch::new(); batch.put(b”user:2”, b”Grace”); batch.put(b”user:3”, b”Linus”); batch.delete(b”user:1”); db.write_batch(&batch).await?; for (key, value) in db.scan_prefix(b”user:“).await? { println!( ”{} = {}”, String::from_utf8_lossy(&key), String::from_utf8_lossy(&value) ); } db.close().await?; Ok(()) }
Runnable examples:
- basic : insert, get, update, and remove
- batch_writes : atomic puts and deletes
- range_queries : ordered range and prefix scans
- concurrent : shared access from Tokio tasks
- persistence : paranoid WAL recovery
- configuration : cache, memtable, and compression options 可运行示例:
- basic : 插入、获取、更新和删除
- batch_writes : 原子性写入和删除
- range_queries : 有序范围和前缀扫描
- concurrent : Tokio任务的共享访问
- persistence : 严格的WAL恢复
- configuration : 缓存、内存表和压缩选项
| Preset | Acknowledgement boundary | Use case |
|---|---|---|
| DbOptions::fast() | In-memory visibility; no WAL | Caches and reproducible data |
| DbOptions::durable() | Appended to the WAL without a per-write sync | Process-crash recovery; recommended default |
| DbOptions::paranoid() | WAL group completed sync_all before return | Strongest mode, subject to filesystem/device guarantees |
| 预设选项: | ||
| 预设 | 确认边界 | 使用场景 |
| --- | --- | --- |
| DbOptions::fast() | 内存可见性;无WAL | 缓存和可重现数据 |
| DbOptions::durable() | 写入WAL但不每次同步 | 进程崩溃恢复;推荐默认值 |
| DbOptions::paranoid() | 返回前完成WAL组同步 | 最强模式,受文件系统/设备保证限制 |
One open Db or Engine exclusively owns its data directory. Use close() or close_with_status() for a clean shutdown; dropping a handle is not a clean shutdown contract. 一个打开的Db或Engine实例独占其数据目录所有权。使用close()或close_with_status()进行优雅关闭;直接丢弃句柄不符合优雅关闭约定。
Keys and values are arbitrary byte sequences supplied through AsRef<[u8]>; strings need to be encoded by the caller. Mutation APIs copy their inputs before returning. Point and collecting reads return owned Vec<u8> values. An empty value is valid data and is distinct from a deleted key. 键和值是通过AsRef<[u8]>提供的任意字节序列;字符串需要由调用者编码。修改API在返回前会复制其输入。点查询和收集读取返回拥有的Vec<u8>值。空值是有效数据,与已删除键不同。
| API | Parameters | Result and behavior |
|---|---|---|
| Db::open(path) | path: AsRef<Path> | Opens or creates the directory with DbOptions::durable() . The open handle exclusively owns the directory. |
| Db::open_with_options(path, options) | Database path and a DbOptions value | Opens with explicit durability, memory, cache, and compression settings. Rejects contradictory settings such as sync_writes = true with the WAL disabled. |
| DbOptions::fast() | None | Returns the no-WAL preset. |
| DbOptions::durable() | None | Returns the process-crash-recoverable WAL preset. |
| DbOptions::paranoid() | None | Returns the sync-before-acknowledgement preset. |
| options.with_compression(compression) | A Compression variant | Builder-style update that returns the modified options. |
| API列表: | ||
| API | 参数 | 结果和行为 |
| --- | --- | --- |
| Db::open(path) | path: AsRef<Path> | 使用DbOptions::durable()打开或创建目录。打开的句柄独占目录所有权。 |
| Db::open_with_options(path, options) | 数据库路径和DbOptions值 | 使用显式持久性、内存、缓存和压缩设置打开。拒绝矛盾设置,如WAL禁用时sync_writes=true。 |
| DbOptions::fast() | 无 | 返回无WAL预设。 |
| DbOptions::durable() | 无 | 返回进程崩溃可恢复的WAL预设。 |
| DbOptions::paranoid() | 无 | 返回确认前同步预设。 |
| options.with_compression(compression) | Compression变体 | 构建器风格更新,返回修改后的选项。 |
All presets start with a 64 MiB memtable, a 64 MiB block cache, and LZ4 compression. Their public fields can be adjusted before opening: 所有预设初始配置为64MB内存表、64MB块缓存和LZ4压缩。其公共字段可在打开前调整:
| DbOptions field | Meaning |
|---|---|
| wal_enabled: bool | Append mutations to the WAL. Disabling it permits process-crash data loss until a successful flush or close. |
| sync_writes: bool | Await a WAL sync barrier before acknowledging each mutation group. Requires wal_enabled . |
| memtable_size: usize | Approximate in-memory byte threshold that triggers a memtable rotation and background flush. |
| block_cache_size: usize | Decompressed SSTable block-cache budget in bytes. Set to 0 to disable the cache. |
| compression: Compression | SSTable compression for newly written data: Lz4 ,Snappy ,Zstd , orNone . Existing tables retain their encoded format. |
| DbOptions字段说明: | |
| 字段 | 含义 |
| --- | --- |
| wal_enabled: bool | 将变更追加到WAL。禁用时允许进程崩溃导致数据丢失,直到成功刷新或关闭。 |
| sync_writes: bool | 在确认每个变更组前等待WAL同步屏障。需要wal_enabled启用。 |
| memtable_size: usize | 触发内存表轮换和后台刷新的近似内存字节阈值。 |
| block_cache_size: usize | 解压缩SSTable块缓存的字节预算。设为0可禁用缓存。 |
| compression: Compression | 新写入数据的SSTable压缩方式:Lz4、Snappy、Zstd或None。现有表保留其编码格式。 |
| API | Parameters | Returns and semantics |
|---|---|---|
| insert(key, value) | Byte-like key and value | Result<()> . Inserts or replaces the key. The selected durability boundary is reached before success. |
| insert_many(entries) | Any iterator of (key, value) pairs | Result<()> . Copies the full iterator and applies entries in order; the last duplicate key wins. This is a bulk API, not one atomic visibility transition. |
| get(key) | Byte-like key | Result<Option<Vec<u8>>> . ReturnsNone for missing or deleted keys andSome(Vec::new()) for a stored empty value. |
| remove(key) | Byte-like key | Result<()> . Writes a tombstone; deleting a missing key is allowed. |
| contains_key(key) | Byte-like key | Result<bool> . Resolves the same state asget and currently incurs its value allocation. |
| write_batch(batch) | ||
| 核心API: | ||
| API | 参数 | 返回值和语义 |
| --- | --- | --- |
| insert(key, value) | 字节型键和值 | Result<()>。插入或替换键值。成功前达到选定的持久性边界。 |
| insert_many(entries) | (key, value)对的任意迭代器 | Result<()>。复制完整迭代器并按顺序应用条目;最后出现的重复键生效。这是批量API,非原子可见性转换。 |
| get(key) | 字节型键 | Result<Option<Vec<u8>>>。对于缺失或删除的键返回None,存储的空值返回Some(Vec::new())。 |
| remove(key) | 字节型键 | Result<()>。写入墓碑标记;允许删除不存在的键。 |
| contains_key(key) | 字节型键 | Result<bool>。解析与get相同的状态,当前会产生值分配开销。 |
| write_batch(batch) |