
CommunityNews
Rethinking Sanakirja, a Rust database engine with fast clones
My last post about Sanakirja sparked a few really constructive discussions, and made me realise that people still cared about the problem of on-disk key-value stores, as unfancy as that problem may sound. This post looks back on some design mistakes I’ve made when I wrote it, and includes benchmarks showing it’s now faster than the fastest equivalent C library.
Why?
A long time ago, Pijul was using LMDB as its backend, with a number of fundamental limitations, including:
- Being restricted to two datatypes: an array of B+ trees where keys are byte strings and values are either (1) byte strings or (2) B+ trees where keys are bytestrings and values are zero-sized. In Rust terminology, this is equivalent to roughly 500 tables, where a table is either
BTreeMap<&[u8], &[u8]>
orBTreeMap<&[u8], BTreeMap<&[u8], ()>>
.- Being written in C, meaning that it is potentially fast, but hard to extend in any nontrivial way. About the “fast” part, my benchmarks show that indeed, it is quite fast — just not as fast as a carefully-designed Rust version.
- More importantly, I needed to fork tables efficiently, without copying anything. This was especially important back then, when Pijul tables for small repositories often weighed dozens of megabytes. It may be slightly less relevant now, but now that it’s there, there is no reason not to use it.This is implemented with an extra table storing reference counts of each page that is referenced at least twice (in order to avoid infinite recursions, the table itself isn’t clonable, and therefore all its pages are referenced once).
This thread was posted by one of our members via one of our news source trackers.
Popular Backend topics

Greetings from Membrane Framework team!
Check out our case study based on our latest projects at Software Mansion.
https://blog.swmansi...
New

Why Zig When There is Already C++, D, and Rust?
No hidden control flow
No hidden allocations
First-class support for no standard library...
New

Rails Best Practices I.
Today I share some of my favorite practices applicable to Ruby on Rails (and to web development on small teams g...
New

Not had time to read it yet but this looks like a good interview…
Our friend Yukihiro Matsumoto, creator of the Ruby programming langua...
New

Everyone outside of tech has heard of JavaScript, Java, Python, Ruby and even .Net, but few if any have heard of F#. However, F# may be o...
New

New

We take a deeper dive with Nathan Long into IOLists in Elixir. We cover what they are, how they work, the power they have when concatenat...
New

This was posted on the Elixir Forum and thought it was worth sharing here!
I love how the excitement of the author shines through and I ...
New

In episode 83 of Thinking Elixir, We talk with Isaac Yonemoto about the Zig language and his Zigler Elixir library. We learn where Zig ca...
New

Hi everyone :wave: I’m excited to share an article detailing how we have reorganized our Elixir/Phoenix project’s directory structure.
W...
New
Other popular topics

New

Rust is an exciting new programming language combining the power of C with memory safety, fearless concurrency, and productivity boosters...
New

I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me.
I think the Planck and the Preonic (o...
New

Seems like a lot of people caught it - just wondered whether any of you did?
As far as I know I didn’t, but it wouldn’t surprise me if I...
New

Here’s the story how one of the world’s first production deployments of LiveView came to be - and how trying to improve it almost caused ...
New

Hi folks,
I don’t know if I saw this here but, here’s a new programming language, called Roc
Reminds me a bit of Elm and thus Haskell. ...
New

If you want a quick and easy way to block any website on your Mac using Little Snitch simply…
File > New Rule:
And select Deny, O...
New

Author Spotlight:
Karl Stolley
@karlstolley
Logic! Rhetoric! Prag! Wow, what a combination. In this spotlight, we sit down with Karl ...
New

Author Spotlight:
Bruce Tate
@redrapids
Programming languages always emerge out of need, and if that’s not always true, they’re defin...
New

This is a very quick guide, you just need to:
Download LM Studio: https://lmstudio.ai/
Click on search
Type DeepSeek, then select the o...
New
Categories:
Sub Categories:
Popular Portals
- /elixir
- /rust
- /ruby
- /wasm
- /erlang
- /phoenix
- /keyboards
- /rails
- /js
- /python
- /security
- /go
- /swift
- /vim
- /clojure
- /emacs
- /java
- /haskell
- /onivim
- /svelte
- /typescript
- /crystal
- /c-plus-plus
- /kotlin
- /tailwind
- /gleam
- /react
- /ocaml
- /elm
- /flutter
- /vscode
- /ash
- /opensuse
- /centos
- /php
- /deepseek
- /html
- /scala
- /zig
- /textmate
- /sublime-text
- /debian
- /nixos
- /lisp
- /agda
- /react-native
- /kubuntu
- /arch-linux
- /revery
- /ubuntu
- /django
- /spring
- /manjaro
- /diversity
- /lua
- /nodejs
- /julia
- /slackware
- /c
- /neovim