Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

dimitarvp
Backend>Blogs/Talks

Steve Klabnik, one of Rust's creators, Twitch.TV stream
Apparently he decided to live-stream how he’s going to create a semver library.
twitch.tv
/rust#video#generaluage-news
12 1369 5
New
First poster: bot
Backend>Blogs/Talks

An Introduction to Crystal
One of my favourite programming languages in the last few years has been Crystal. While the language has not yet reached its 1.0 version,...
blog.oisinaylward.me
/crystal
0 1490 0
New
First poster: bot
Backend>Blogs/Talks

I finally escaped Node (and you can too)
Creation vs. Evolution Consider the history of Elixir: first you take Erlang, which was invented by Joe Armstrong and team to solve the ...
acco.io
/elixir/nodejs
1 1439 0
New
CommunityNews
Backend>Blogs/Talks

Functional Programming in Go with Generics
Functional programming is an increasing popular programming paradigm with many languages building or already supporting it. Go already su...
ani.dev
/go#programming
4 1625 1
New
First poster: bot
Backend>Blogs/Talks

From Julia to Rust
I’ve been more serious about learning Rust recently, after dragging on with passive learning for a while. My first real programming langu...
miguelraz.github.io
/julia/rust
2 1623 0
New
AstonJ
Backend>Blogs/Talks

This is the Future of Webdev. Crazy, Right?
This was posted on the Elixir Forum and thought it was worth sharing here! I love how the excitement of the author shines through and I ...
suzdalnitski.medium.com
#elixr
12 1328 4
New
axelson
Backend>Blogs/Talks

Hot Reloading: A Faster Way to Develop Modern Apps
I describe how we use Hot Reloading with Webpack to develop faster and show how to integrate Webpack 5, webpack-dev-server, and Phoenix f...
felt.com
/elixir#blog-post
9 1472 3
New
brainlid
Backend>Blogs/Talks

ThinkingElixir 092 - Temple with Mitchell Hanberg
In episode 92 of Thinking Elixir, we talk with Mitchell Hanberg and learn about why he created the alternate Phoenix templating language ...
podcast.thinkingelixir.com
/elixir#podcasts#testing#browser/html#dev-news-source
4 1327 0
New
StuntProgrammer
Backend>Blogs/Talks

Media Storage and CDN with Phoenix and Elixir
In building lofi.limo, media storage and distribution naturally came up. I have songs, announcements, and background image loops which I ...
lofi.limo
/elixir#article
20 1334 5
New
mtmattei
Backend>Blogs/Talks

State of WebAssembly 2024-2025
For the past few years, Safari has been putting in a lot of effort to enhance its WebAssembly support and 2024 was no exception… I believ...
platform.uno
#web/wasm
1 1468 0
New

Other popular topics Top

Devtalk
General Dev>Dev Chat

What dev-related stuff have you been up to?
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
#community
1063 23050 405
New
PragmaticBookshelf
Backend>Learning Resources

Distributed Services with Go
Take your Go skills to the next level by learning how to design, develop, and deploy a distributed service. Start from the bare essential...
pragprog.com
#pragprog/go#published-book/book-distributed-services-with-go
1 4310 0
New
dasdom
General Dev>Dev Chat

Standing Desks
No chair. I have a standing desk. This post was split into a dedicated thread from our thread about chairs :slight_smile:
#workspace#opinions
177 9886 77
New
AstonJ
General Dev>Hardware

Planck vs Preonic vs Subatomic (Keyboards)
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
/keyboards#mechanical-keyboards#ortholinear#planck#preonic
105 17596 47
New
Margaret
General Dev>Dev Chat

PragProg’s Medium Posts
Hello everyone! This thread is to tell you about what authors from The Pragmatic Bookshelf are writing on Medium.
#pragprog#blog-post
1147 29994 760
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 11719 28
New
DevotionGeo
General Dev>Questions

Do you prefer regular mechanical keyboards or low profile mechanical keyboards and why?
I have always used antique keyboards like Cherry MX 1800 or Cherry MX 8100 and almost always have modified the switches in some way, like...
/keyboards#mechanical-keyboards
27 3877 9
New
PragmaticBookshelf
Backend>Learning Resources

Server-Driven Web Apps with htmx
Build modern server-driven web applications using htmx. Whatever programming language you use, you’ll write less (and cleaner) code. ...
pragprog.com
#pragprog#web-development#published-book/book-server-driven-web-apps-with-htmx
6 5257 3
New
CommunityNews
General Dev>In The News

Review of Linux on Minisforum V3 AMD Ryzen Tablet
A Brief Review of the Minisforum V3 AMD Tablet. Update: I have created an awesome-minisforum-v3 GitHub repository to list information fo...
mudkip.me
#linux#review#amd
0 4635 0
New
AstonJ
Backend>Questions

Psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
#macos/rails/postgresql
1 5553 1
New
Back to top

Sponsor Spotlight

EEF

Supporting innovation across the BEAM ecosystem.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap