Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

dimitarvp
Backend>Blogs/Talks

Steve Klabnik, one of Rust's creators, Twitch.TV stream
Apparently he decided to live-stream how he’s going to create a semver library.
twitch.tv
/rust#video#generaluage-news
12 1369 5
New
First poster: iPaul
Backend>Blogs/Talks

The Future of Clojure
thoughtworks.com
/clojure#podcasts
3 1823 2
New
First poster: bot
Backend>Blogs/Talks

How NOT to Teach Recursion
We all know how to teach recursion. We’ve done it for decades. We pick some honored, time-tested examples—Fibonacci numbers and factorial...
parentheticallyspeaking.org
/elixir/erlang/gleam/lfe/caramel
1 1525 0
New
First poster: AstonJ
Backend>Blogs/Talks

Ten years without Elixir
Ten years without Elixir. I never got into Elixir, largely because it looked like Ruby. I was a Rubyist for a good while, spent time and...
blog.cretaria.com
/elixir
9 1705 3
New
paulanthonywilson
Backend>Blogs/Talks

Blog on Content Security Policy configuration (in Phoenix with LiveView)
I had a bit of a mini-adventure following Sobelow’s advice on adding a CSP to a Phoenix App. If you want to follow along, or want to add ...
furlough.merecomplexities.com
/elixir/phoenix#blog-post/security
3 1860 0
New
CommunityNews
Backend>Blogs/Talks

Clojure by Example (2015)
I don’t like reading thick O’Reilly books when I start learning new programming languages. Rather, I like starting by writing small and d...
kimh.github.io
/clojure
7 1527 1
New
AstonJ
Backend>Blogs/Talks

This is the Future of Webdev. Crazy, Right?
This was posted on the Elixir Forum and thought it was worth sharing here! I love how the excitement of the author shines through and I ...
suzdalnitski.medium.com
#elixr
12 1328 4
New
elbrujohalcon
Backend>Blogs/Talks

Inaka / TDD, Coverage and Why Testing Exceptions
Another week, another oldies-but-goldies post… This one about Test Driven Development.
tealfeed.com
/erlang#article#test#tdd
4 1421 0
New
RudManusachi
Backend>Blogs/Talks

SQL CASE with Ecto
Hi there! Recently I was playing around with extracting and updating data in the DB and for fun challenged myself to try to implement a ...
manusachi.com
/elixir#blog-post#postgres#blog-posts#ecto#tba-blog-post
8 1644 1
New
tonyxrandall
Backend>Blogs/Talks

How to leverage functional programming in Kotlin to write better, cleaner code
As DoorDash transitioned from Python monolith to Kotlin microservices, our engineering team was presented with a lot of opportunities to ...
doordash.engineering
/kotlin#article#functional-programming
5 3159 1
New

Other popular topics Top

Devtalk
General Dev>Dev Chat

HELLO WORLD (Introductions thread!)
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
#community
481 6762 116
New
PragmaticBookshelf
General Dev>Learning Resources

Seven More Languages in Seven Weeks
Learn from the award-winning programming series that inspired the Elixir language, and go on a step-by-step journey through the most impo...
pragprog.com
#pragprog/elixir/julia/lua#published-book#factor/elm#minikanren/idris/book-seven-more-languages-in-seven-weeks
4 5862 0
New
AstonJ
General Dev>Code Editors

Poll: Which code editor do you use?
You might be thinking we should just ask who’s not using VSCode :joy: however there are some new additions in the space that might give V...
#community#polls/vim/emacs#code-editors/vscode#notepad/sublime-text#atom/textmate#codespaces#brackets/onivim#geany
121 5796 61
New
AstonJ
General Dev>Hardware

Seen any cool new keyboards?
We have a thread about the keyboards we have, but what about nice keyboards we come across that we want? If you have seen any that look n...
/keyboards#mechanical-keyboards
49 5910 39
New
AstonJ
General Dev>Hardware

Custom keyboard keycaps
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
#hardware/keyboards#keycaps#mechanical-keyboards
15 11086 19
New
PragmaticBookshelf
Backend>Learning Resources

Concurrent Data Processing in Elixir
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
pragprog.com
#pragprog/elixir#published-book/book-concurrent-data-processing-in-elixir
78 6059 24
New
PragmaticBookshelf
Backend>Learning Resources

Effective Haskell
Build efficient applications that exploit the unique benefits of a pure functional language, learning from an engineer who uses Haskell t...
pragprog.com
#pragprog/haskell#published-book/book-effective-haskell
15 10218 1
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 11719 28
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Peter Ullrich (Author) Interview and AMA!
Author Spotlight: Peter Ullrich @PJUllrich Data is at the core of every business, but it is useless if nobody can access and analyze ...
/elixir/phoenix/book-building-table-views-with-phoenix-liveview
72 4765 21
New
AnfaengerAlex
Android>Questions

Unresolved Reference to android in build.gradle.kts – Beginner Issue
Hello, I’m a beginner in Android development and I’m facing an issue with my project setup. In my build.gradle.kts file, I have the foll...
#binding
0 7460 2
New
Back to top

Sponsor Spotlight

Manning

Develop your skills with books, videos, and courses.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap