Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

dimitarvp
Backend>Blogs/Talks

Steve Klabnik, one of Rust's creators, Twitch.TV stream
Apparently he decided to live-stream how he’s going to create a semver library.
twitch.tv
/rust#video#generaluage-news
12 1369 5
New
wolf4earth
Backend>Blogs/Talks

How we split Plug.SSL to control SSL redirection
dev.betterdoc.org
/elixir#elixir-plug#ssl#https
5 1651 2
New
ErlangSolutions
Backend>Blogs/Talks

OCaml on the BEAM
Interested in a blazingly fast type checker with 25 years of investment, delivered on the BEAM? Check out Caramel, an exciting project fr...
www2.erlang-solutions.com
/erlang/ocaml#beam/caramel
5 1959 2
New
AstonJ
Backend>Blogs/Talks

Industry vs Academia - Francesco Cesarini & Simon Thompson at Lambda Days 2021
Just listening to this now… Totally agree with @FrancescoC’s and @thompson_si’s comment “learn to learn” :sunglasses: In our talk we’...
youtube.com
/elixir/erlang#talk
7 1496 1
New
First poster: bot
Backend>Blogs/Talks

Phantom Types in Gleam
In this post we’re going to be looking at a more advanced use of Gleam’s type system, known as phantom types. Hopefully by the end of thi...
blog.pd-andy.dev
/gleam#types
0 1579 0
New
AstonJ
Backend>Blogs/Talks

This is the Future of Webdev. Crazy, Right?
This was posted on the Elixir Forum and thought it was worth sharing here! I love how the excitement of the author shines through and I ...
suzdalnitski.medium.com
#elixr
12 1328 4
New
First poster: AstonJ
Backend>Blogs/Talks

Ruby Structs
Ruby’s Struct is one of several powerful core classes which is often overlooked and under utilized compared to the more popular Hash clas...
alchemists.io
/ruby
20 1993 13
New
MarcinKasprowicz
Backend>Blogs/Talks

Elixir for JavaScript developers: first impressions
Elixir language viewed from the perspective of a JavaScript developer. I compared selected aspects of the two languages and touched on to...
medium.com
/elixir#blog-post#functional-programming/js
4 1188 3
New
mtmattei
Backend>Blogs/Talks

State of WebAssembly 2024-2025
For the past few years, Safari has been putting in a lot of effort to enhance its WebAssembly support and 2024 was no exception… I believ...
platform.uno
#web/wasm
1 1468 0
New
brainlid
Backend>Blogs/Talks

Thinking Elixir 264 - Hot Reload In Dev and QA Bottlenecks
Episode 264 of Thinking Elixir. News includes Phoenix v1.8 shipping with an AGENTS markdown file for new apps to enhance coding agent exp...
youtube.com
#ai/elixir#podcasts/wasm#llm
1 630 0
New

Other popular topics Top

PragmaticBookshelf
General Dev>Learning Resources

The Pragmatic Programmer, 20th Anniversary Edition
Andy and Dave wrote this influential, classic book to help their clients create better software and rediscover the joy of coding. Almost ...
pragprog.com
#pragprog#published-book/book-the-pragmatic-programmer-20th-anniversary-edition
4 4782 0
New
PragmaticBookshelf
Game Dev>Learning Resources

Apple Game Frameworks and Technologies
Design and develop sophisticated 2D games that are as much fun to make as they are to play. From particle effects and pathfinding to soci...
pragprog.com
#pragprog#ios#game-dev#macos/swift#published-book#apple/book-apple-game-frameworks-and-technologies
30 7995 10
New
PragmaticBookshelf
Android>Learning Resources

Kotlin and Android Development featuring Jetpack: Build Better, Safer Android Apps
Start building native Android apps the modern way in Kotlin with Jetpack's expansive set of tools, libraries, and best practices. Learn h...
pragprog.com
#pragprog#android#game-dev/kotlin#published-book/book-kotlin-and-android-development-featuring-jetpack
7 5084 1
New
AstonJ
Backend>Chat

How to install Ruby 3 with ASDF
In case anyone else is wondering why Ruby 3 doesn’t show when you do asdf list-all ruby :man_facepalming: do this first: asdf plugin-upd...
/ruby#asdf
11 5961 4
New
AstonJ
General Dev>Dev Chat

Languages Without Garbage Collection
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
#garbage-collection
21 5575 7
New
PragmaticBookshelf
Backend>Learning Resources

C Brain Teasers
25 puzzles that will make you a better C programmer by challenging your knowledge of the language, and explaining the technical details o...
pragprog.com
#pragprog#published-book/c/book-c-brain-teasers
4 3873 2
New
PragmaticBookshelf
Backend>Learning Resources

Ash Framework
Explore the power of Ash Framework by modeling and building the domain for a real-world web application. Rebecca Le @sevenseacat and ...
pragprog.com
#pragprog/elixir#published-book/ash/book-ash-framework
15 7555 9
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 4275 15
New
RobertRichards
Game Dev>Chat

Hair Salon Games for Girls Fun
Hair Salon Games for Girls Fun Girls Hair Saloon game is mainly developed for kids. This game allows users to select virtual avatars to ...
#ios#android#unity#fun
2 3556 1
New
PragmaticBookshelf
General Dev>Learning Resources

Write Better with Vale
Lint your docs like code: turn any style guide into enforceable rules with Vale and publish clear, consistent content every time. ...
pragprog.com
#pragprog#published-book#vale/book-write-better-with-vale
8 9973 5
New
Back to top

Sponsor Spotlight

WyewWorks

Enabling companies to succeed by building software people love.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap