Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

CommunityNews
Backend>Blogs/Talks

Optimization - Making Rust Code Go Brrrr
aspenuwu.me
/go/rust#code
2 1429 1
New
wolf4earth
Backend>Blogs/Talks

How we split Plug.SSL to control SSL redirection
dev.betterdoc.org
/elixir#elixir-plug#ssl#https
5 1651 2
New
First poster: bot
Backend>Blogs/Talks

It’s not legacy code — it’s PHP
It’s not legacy code — it’s PHP. Vimeo has been using PHP in production for over 15 years. Find out how we keep a million lines of PHP i...
medium.com
/php#code
0 1460 0
New
AstonJ
Backend>Blogs/Talks

How to set-up a Ruby and Elixir dev environment on macOS (2021)
Just finished doing a clean install of macOS (which I highly recommend btw!) and have updated my macOS Ruby & Elixir/Erlang dev env s...
astonj.com
/elixir/erlang/ruby/phoenix/nodejs#asdf
3 1889 0
New
First poster: bot
Backend>Blogs/Talks

PHP in 2021
Once a year, I look back at the recent developments in the PHP world, and also look forward to what’s to come. And just like in 2020 and ...
stitcher.io
/php
0 1390 0
New
CommunityNews
Backend>Blogs/Talks

Structural typing in Rust
Have you ever wanted to write a structurally typed function in Rust? Do you spend a lot of time and effort getting your Rust struct s jus...
beachape.com
/rust#typing
5 1344 1
New
tonyxrandall
Backend>Blogs/Talks

Migrating From Python to Kotlin for Our Backend Services (DoorDash)
When DoorDash approached the limits of what our Django-based monolithic codebase could support, we needed to design a new stack that woul...
doordash.engineering
/kotlin/python#blog-post#backend
3 3489 0
New
elbrujohalcon
Backend>Blogs/Talks

Inaka / TDD, Coverage and Why Testing Exceptions
Another week, another oldies-but-goldies post… This one about Test Driven Development.
tealfeed.com
/erlang#article#test#tdd
4 1421 0
New
brainlid
Backend>Blogs/Talks

ThinkingElixir 092 - Temple with Mitchell Hanberg
In episode 92 of Thinking Elixir, we talk with Mitchell Hanberg and learn about why he created the alternate Phoenix templating language ...
podcast.thinkingelixir.com
/elixir#podcasts#testing#browser/html#dev-news-source
4 1327 0
New
mudasobwa
Backend>Blogs/Talks

Long-lived Process and State Recovery After Crashes
Peeper is the tiny library to preserve state across GenServer crashes/restarts. Works as an almost drop-in substitute for GenServer, sui...
rocket-science.ru
/elixir#blog-post#opensource
3 964 0
New

Other popular topics Top

DevotionGeo
Backend>Questions

Can someone explain the -t option/flag in docker run command?
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
#docker
7 10261 2
New
AstonJ
General Dev>Hardware

Poll: Which keyboard layout do you use?
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
colemakmods.github.io
#polls/keyboards
10 6048 11
New
AstonJ
General Dev>Code Editors

Onivim 2 Code Editor
Thanks to @foxtrottwist’s and @Tomas’s posts in this thread: Poll: Which code editor do you use? I bought Onivim! :nerd_face: https://on...
#code-editors/onivim/revery
88 6144 32
New
AstonJ
General Dev>Hardware

BIIP MT3 Extended 2048 Custom Keycap Set (Drop)
This looks like a stunning keycap set :orange_heart: A LEGENDARY KEYBOARD LIVES ON When you bought an Apple Macintosh computer in the e...
/keyboards#apple#keycaps#mechanical-keyboards
14 6713 7
New
AstonJ
General Dev>Dev Chat

How fast do you type? Check your WPM here!
Do the test and post your score :nerd_face: :keyboard: If possible, please add info such as the keyboard you’re using, the layout (Qw...
typing-speed-test.aoeu.eu
/keyboards
82 7682 31
New
dimitarvp
macOS>Chat

My thoughts on macOS vs Linux
Small essay with thoughts on macOS vs. Linux: I know @Exadra37 is just waiting around the corner to scream at me “I TOLD YOU SO!!!” but I...
#macos#linux
166 9730 69
New
Exadra37
Linux>Chat

RancherOS is in end of life
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
#linux#rancheros
10 6358 6
New
PragmaticBookshelf
Backend>Learning Resources

Programming WebRTC
Use WebRTC to build web applications that stream media and data in real time directly from one user to another, all in the browser. ...
pragprog.com
#pragprog#published-book/js#webrtc/book-programming-webrtc
27 6969 6
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Jamis Buck (Author) Interview and AMA!
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
#author-spotlight/ruby/book-the-ray-tracer-challenge/book-mazes-for-programmers
21 6352 9
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Erin Dees (Author) Interview and AMA!
Author Spotlight Erin Dees @undees Welcome to our new author spotlight! We had the pleasure of chatting with Erin Dees, co-author of ...
#author-spotlight/ruby/rails/book-effective-testing-with-rspec-3/book-seven-more-languages-in-seven-weeks/book-cucumber-recipes#rspec
24 4247 11
New
Back to top

Sponsor Spotlight

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap