The Marginalia Search index has been largely rewritten to perform much better, using new data structures designed to make better use of modern hardware. This post will cover the new design, and will also touch upon some of the unexpected and unintuitive performance characteristics of NVMe SSDs when it comes to read sizes.
The index is already fairly large, but can sometimes feel smaller than it is, and paradoxically, query performance is a big part of why.

Read in full here: