Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

First poster: bot
Backend>Blogs/Talks

Introduction to Postgrest
Part 1: Introduction to Postgrest. In Codd, we trust In the field of Computer Science and Engineering, few things come close to the dura...
samkhawase.com
#postgres
11 1820 5
New
AstonJ
Backend>Blogs/Talks

How to set-up a Ruby and Elixir dev environment on macOS (2021)
Just finished doing a clean install of macOS (which I highly recommend btw!) and have updated my macOS Ruby & Elixir/Erlang dev env s...
astonj.com
/elixir/erlang/ruby/phoenix/nodejs#asdf
3 1889 0
New
prajaut
Backend>Blogs/Talks

Golang Error Handling blog post
Being a part of the tech industry, it would be good to share thoughts on specific technologies. Having surrounded by skilled and experie...
bacancytechnology.com
/go
7 1898 2
New
First poster: bot
Backend>Blogs/Talks

PHP in 2021
Once a year, I look back at the recent developments in the PHP world, and also look forward to what’s to come. And just like in 2020 and ...
stitcher.io
/php
0 1390 0
New
First poster: bot
Backend>Blogs/Talks

Zig Makes Rust Cross-compilation Just Work
This post is a spiritual successor to Loris Cro’s Go cross-compilation. The encounter During a recent stage 2 meeting Jakub Konka wanted...
actually.fyi
/rust/zig
2 1524 0
New
elbrujohalcon
Backend>Blogs/Talks

How do List Functions Fail in Erlang?
A long time ago, I wrote an article about The Asymmetry of ++, thanks to Fede Bergero’s findings. Let’s add a few more asymmetries to th...
medium.com
/erlang#blog-post
17 1659 5
New
elbrujohalcon
Backend>Blogs/Talks

Spot the Difference
Erlang is famous for its introspecting powers. You can get a lot of information about the processes running in your nodes without any ext...
medium.com
/elixir/erlang#blog-post/gleam#efene
35 1952 6
New
First poster: bot
Backend>Blogs/Talks

A Clojure processor that's almost a drop-in replacement for php-fpm behind Nginx
Too long have we hustled to deploy Clojure websites. Too long have we spun up one server instance per site. Too long have reminisced abou...
github.com
/clojure/php#nginx
0 1472 0
New
tonyxrandall
Backend>Blogs/Talks

Migrating From Python to Kotlin for Our Backend Services (DoorDash)
When DoorDash approached the limits of what our Django-based monolithic codebase could support, we needed to design a new stack that woul...
doordash.engineering
/kotlin/python#blog-post#backend
3 3489 0
New
brainlid
Backend>Blogs/Talks

Thinking Elixir 244 - Running Python in Elixir?
Episode 244 of Thinking Elixir. News includes the release of Elixir 1.18.2 with various enhancements and bug fixes, a new experimental SQ...
youtube.com
/elixir/python#podcasts#livebook
1 841 0
New

Other popular topics Top

AstonJ
Science/Tech>Tech Chat

What are you watching?
Or looking forward to? :nerd_face:
#community
503 14512 277
New
DevotionGeo
Backend>Questions

Can someone explain the -t option/flag in docker run command?
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
#docker
7 10261 2
New
AstonJ
General Dev>Code Editors

Poll: Which code editor do you use?
You might be thinking we should just ask who’s not using VSCode :joy: however there are some new additions in the space that might give V...
#community#polls/vim/emacs#code-editors/vscode#notepad/sublime-text#atom/textmate#codespaces#brackets/onivim#geany
121 5796 61
New
AstonJ
General Dev>Hardware

Poll: Which keyboard layout do you use?
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
colemakmods.github.io
#polls/keyboards
10 6048 11
New
AstonJ
Frontend>Chat

Online Hand to eye coordination test
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
#online-tools
4 4562 1
New
PragmaticBookshelf
Backend>Learning Resources

Programming Ruby 3.2 (5th Edition)
Programming Ruby is the most complete book on Ruby, covering both the language itself and the standard library as well as commonly used t...
twitter.com
#pragprog/ruby/rails#published-book/book-programming-ruby-3-2-5th-edition
28 8550 13
New
First poster: AstonJ
General Dev>In The News

Jan: An open source alternative to ChatGPT that runs on the desktop
Jan | Rethink the Computer. Jan turns your computer into an AI machine by running LLMs locally on your computer. It’s a privacy-focus, l...
jan.ai
#desktop#chatgpt
4 5652 4
New
sir.laksmana_wenk
Game Dev>Questions

I want to learn how make a game, but where should I start?
I’m able to do the “artistic” part of game-development; character designing/modeling, music, environment modeling, etc. However, I don’t...
#game-dev
15 4965 9
New
AstonJ
AI>In The News

DeepSeek (671B) running on a cluster of 8 Mac Mini Pros with 64GB RAM each
This is cool! DEEPSEEK-V3 ON M4 MAC: BLAZING FAST INFERENCE ON APPLE SILICON We just witnessed something incredible: the largest open-s...
#ai#macs/deepseek
0 6695 1
New
NewsBot
Backend>Official News

Node.js v22.14.0 released!
Node.js v22.14.0 has been released. Link: Release 2025-02-11, Version 22.14.0 'Jod' (LTS), @aduh95 · nodejs/node · GitHub
github.com
/nodejs#official-news
0 4251 0
New
Back to top

Sponsor Spotlight

coders51

We build reliable cloud platforms for business-critical systems.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap