Backend Developer Forum>Blogs/Articles/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
#sqlite/python/rust
0 808 0

First Post!

bot

bot

Corresponding tweet for this thread:

Share link for this tweet.

Post #2

Popular Backend topics Top

PragmaticBookshelf
Backend Developer Forum>Learning Resources

Programming Phoenix 1.4
Don’t accept the compromise between fast and beautiful: you can have it all. Phoenix creator Chris McCord, Elixir creator José Valim, and...
/phoenix#pragprog#published-book/book-programming-phoenix-1-4/elixir
16 1530 3
New
PragmaticBookshelf
Backend Developer Forum>Learning Resources

Craft GraphQL APIs in Elixir with Absinthe
Your domain is rich and interconnected, and your API should be, too. Upgrade your web API to GraphQL, using flexible queries to empower y...
/elixir#pragprog#published-book#api/book-craft-graphql-apis-in-elixir-with-absinthe
22 1805 10
New
PragmaticBookshelf
Backend Developer Forum>Learning Resources

Testing Elixir
Write Elixir tests that you can be proud of. Dive into Elixir’s test philosophy and gain mastery over the terminology and concepts that u...
/elixir#pragprog#published-book/book-testing-elixir
33 2419 8
New
Kurisu
Backend Developer Forum>Questions/Help

Ruby on Rails book recommendations please
Hello and happy new year! I would like to buy a Ruby On Rails ebook for learning purpose. What would be the ROR equivalent of “Programm...
/ruby/rails#book
17 1559 7
New
PragmaticBookshelf
Backend Developer Forum>Learning Resources

Learn to Program, Third Edition
It's easier to learn how to program a computer than it has ever been before. Now everyone can learn to write programs for themselves—no p...
#pragprog/ruby#published-book/book-learn-to-program-third-edition
11 1821 5
New
brennan
Backend Developer Forum>Questions/Help

Are there any good resources of using CQRS/EventSourcing in an Elixir/Phoenix application?
Are there any good resources of using CQRS/EventSourcing in an Elixir/Phoenix application?
/elixir/phoenix#cqrs#event-sourcing
16 1814 5
New
Cellane
Backend Developer Forum>Chat/Discussions

Phoenix 1.6.0 Released
Phoenix 1.6.0 got released last week, with built-in authentication and mailer generators, a whole new HEEx (HTML-aware Embedded Elixir) e...
phoenixframework.org
/elixir/phoenix#framework-release-news
87 1952 29
New
First poster: bot
Backend Developer Forum>News

Node.js v14.19.0 released!
Node.js v14.19.0 has been released. Link: Release 2022-02-01, Version 14.19.0 'Fermium' (LTS), @richardlau · nodejs/node · GitHub
github.com
/nodejs#official-news
0 1407 0
New
PragmaticBookshelf
Backend Developer Forum>Learning Resources

Programming Ruby 3.2 (5th Edition)
Programming Ruby is the most complete book on Ruby, covering both the language itself and the standard library as well as commonly used t...
#pragprog/ruby/rails#published-book/book-programming-ruby-3-2-5th-edition
28 2052 13
New
ManningBooks
Backend Developer Forum>Learning Resources

Elixir in Action, Third Edition
Fully updated to Elixir 1.14, this authoritative bestseller reveals how Elixir tackles problems of scalability, fault tolerance, and high...
#manning/elixir#development#meap#published-book
19 1552 11
New

Other popular topics Top

malloryerik
Frontend Developer Forum>Chat/Discussions

Svelte - what do you think about it?
Any thoughts on Svelte? Svelte is a radical new approach to building user interfaces. Whereas traditional frameworks like React and Vue...
#opinions/svelte
54 3345 17
New
wolf4earth
Health & Diet

How do you keep fit and healthy?
@AstonJ prompted me to open this topic after I mentioned in the lockdown thread how I started to do a lot more for my fitness. https://f...
#fitness#heath
193 4292 82
New
DevotionGeo
Backend Developer Forum>Chat/Discussions

Would you use Erlang now when there is Elixir?
Why, if your answer is yes?
/elixir/erlang
167 4195 52
New
AstonJ
Hardware

Seen any cool new keyboards?
We have a thread about the keyboards we have, but what about nice keyboards we come across that we want? If you have seen any that look n...
/keyboards#mechanical-keyboards
44 5023 35
New
AstonJ
Hardware

GMK Serika Keycaps - Serika 2 available to order now!
I have seen the keycaps I want - they are due for a group-buy this week but won’t be delivered until October next year!!! :rofl: The Ser...
/keyboards#keycaps#mechanical-keyboards
9 4181 7
New
Exadra37
Linux Developer Forum>Questions/Help

What is the most minimalist Linux server distro?
I am asking for any distro that only has the bare-bones to be able to get a shell in the server and then just install the packages as we ...
#linux#servers
66 20738 24
New
AstonJ
Health & Diet

Did you manage to avoid covid19?
Seems like a lot of people caught it - just wondered whether any of you did? As far as I know I didn’t, but it wouldn’t surprise me if I...
#covid19
190 3638 79
New
gagan7995
Backend Developer Forum>Questions/Help

Please tell me how to write a query for this in nodejs
API 4 Path: /user/following/ Method: GET Description: Returns the list of all names of people whom the user follows Response [ { ...
/nodejs
7 2856 3
New
First poster: joeb
In The News

Safari now supports File System Access API with private origin
The File System Access API with Origin Private File System. WebKit supports new API that makes it possible for web apps to create, open,...
webkit.org
#safari#api
43 2807 21
New
Help
Game Developer Forum>Questions/Help

Can I use Java to program a game for Nintendo switch?
I am trying to crate a game for the Nintendo switch, I wanted to use Java as I am comfortable with that programming language. Can you use...
/java#nintendo
8 3235 3
New
Back to top

Latest in Backend

Symfony v7.3.0-BETA1, v7.2.6 and v6.4.21 released!
Backend News
Ruby: CVE-2025-43857: DoS vulnerability in net-imap
Backend News
V weekly.2025.18 released!
Backend News
Kotlin v2.1.21-RC2 released!
Backend News
Quarkus 3.22.1 and 3.22.0 released!
Backend News
PostgreSQL: pgAdmin 4 v9.3 Released
Backend News
Kotlin v2.2.0-Beta2 released!
Backend News
Kotlin: Structuring Ktor Projects Using Domain-Driven Design (DDD) Concepts: A Step-by-Step Guide With a Minimalistic CRM
Backend News
PostgreSQL: pgDay Lowlands in Rotterdam - Call For Presentations (CfP) Closing Soon, and the Call for Sponsors is Open!
Backend News
Crystal 1.16.2 is released!
Backend News
PostgreSQL: pg_auto_reindexer v1.4 released
Backend News
Crystal 1.16.2 released!
Backend News
How to Use Kotlin Notebooks for Productive Development
Backend News
PostgreSQL: Streaming Data into Postgres with Estuary
Backend News
Clojure: Introducing core.async.flow
Backend News
Django: New Features GitHub repo and project
Backend News
V weekly.2025.17 released!
Backend News
Julia v1.12.0-beta2 released!
Backend News
Clojure Deref (Apr 25, 2025)
Backend News
R: Sensitivity to C math library and mingw-w64 v12
Backend News

View all threads ❯

Latest (all)

Owen Le Blanc: creator of the first Linux distribution
In The News
Mike Waltz Accidentally Reveals Obscure App the Government Is Using to Archive Signal Messages
In The News
Zoho Halts $700 million Semiconductor Plan
In The News
Symfony v7.3.0-BETA1, v7.2.6 and v6.4.21 released!
Backend News
openSUSE: Tumbleweed Monthly Update - April 2025
Linux News
Creating Beautiful Charts with JRuby and JFreeChart
Frontend In the News
When ChatGPT Broke an Entire Field: An Oral History | Quanta Magazine
In The News
AI Code Review: Should the Author Be The Reviewer?
In The News
Claude can now connect to your world
In The News
Why Fivetran and Census are joining forces | Blog | Fivetran
In The News
React Native v0.79.2 released!
Hybrid News
Ruby: CVE-2025-43857: DoS vulnerability in net-imap
Backend News
V weekly.2025.18 released!
Backend News
Kotlin v2.1.21-RC2 released!
Backend News
"AI-first" is the new Return To Office - Anil Dash
In The News

View all threads ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap