Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

CommunityNews
Backend>Blogs/Talks

Is Zig the Long Awaited C Replacement
Is Zig the Long Awaited C Replacement. Comparison with previous C contenders such as C++, D, Java, C#, Go, Rust and Swift https://erik...
erik-engheim.medium.com
/c/c-plus-plus/rust/swift/zig/v
4 2404 1
New
First poster: wolf4earth
Backend>Blogs/Talks

Understanding Partial Moves in Rust
Understanding Partial Moves in Rust. Partial moves are an interesting but often misunderstood feature of Rust. However, with the right ...
whileydave.com
/rust
8 1614 4
New
First poster: bot
Backend>Blogs/Talks

Quick Overview of Julia language [pdf]
Julia is a scientific programming language that is free and open source.1 It is a relatively new language that borrows inspiration from l...
algorithmsbook.com
/julia
0 1567 0
New
First poster: bot
Backend>Blogs/Talks

Why Zig when there is already C++, D, and Rust?
Why Zig When There is Already C++, D, and Rust? No hidden control flow No hidden allocations First-class support for no standard library...
ziglang.org
/c-plus-plus/rust/zig
0 1788 0
New
First poster: bot
Backend>Blogs/Talks

What's Next for Teal, the typed dialect of Lua
What’s Next for Teal, the typed dialect of Lua - FOSDEM 2021. This is my talk about the latest updates on the Teal programming language,...
youtube.com
/lua
0 1352 0
New
First poster: bot
Backend>Blogs/Talks

PHP in 2021
Once a year, I look back at the recent developments in the PHP world, and also look forward to what’s to come. And just like in 2020 and ...
stitcher.io
/php
0 1390 0
New
First poster: brennan
Backend>Blogs/Talks

Java Is Underhyped
The perspective of an ignorant computer science undergrad It’s likely that you read the title of this post and thought “what is this guy ...
jackson.sh
/java
8 1598 4
New
brainlid
Backend>Blogs/Talks

Safe Ecto Migrations - Recipes and Resources
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
fly.io
/elixir#blog-post#postgres#database
4 1417 0
New
StuntProgrammer
Backend>Blogs/Talks

Media Storage and CDN with Phoenix and Elixir
In building lofi.limo, media storage and distribution naturally came up. I have songs, announcements, and background image loops which I ...
lofi.limo
/elixir#article
20 1334 5
New
brainlid
Backend>Blogs/Talks

Created my Personal AI Fitness Trainer in 2 Days
In a 2 day spike, I created my own Elixir-based AI Personal Fitness Trainer! The surprising part for me was how useful and helpful I foun...
fly.io
#ai/elixir#blog-post#fitness#chatgpt#workout
2 1120 0
New

Other popular topics Top

PragmaticBookshelf
Backend>Learning Resources

Programming Machine Learning
Machine learning can be intimidating, with its reliance on math and algorithms that most programmers don't encounter in their regular wor...
pragprog.com
#pragprog#ai/python#published-book/book-programming-machine-learning#math#algorithms
6 5350 3
New
AstonJ
Science/Tech>Tech Chat

What are you watching?
Or looking forward to? :nerd_face:
#community
503 15149 280
New
DevotionGeo
General Dev>Code Editors

Dendron: a personal knowledge management tool on top of VSCode
/vscode#visual-studio-code
30 8077 9
New
PragmaticBookshelf
Backend>Learning Resources

Programming WebRTC
Use WebRTC to build web applications that stream media and data in real time directly from one user to another, all in the browser. ...
pragprog.com
#pragprog#published-book/js#webrtc/book-programming-webrtc
27 6969 6
New
PragmaticBookshelf
Backend>Learning Resources

Agile Web Development with Rails 7
Rails 7 completely redefines what it means to produce fantastic user experiences and provides a way to achieve all the benefits of single...
pragprog.com
#pragprog#web-development/ruby/rails#published-book/book-agile-web-development-with-rails-7
32 6600 9
New
Help
Game Dev>Questions

Can I use Java to program a game for Nintendo switch?
I am trying to crate a game for the Nintendo switch, I wanted to use Java as I am comfortable with that programming language. Can you use...
/java#nintendo
8 4771 3
New
First poster: bot
General Dev>In The News

Zig now has built-in HTTP server and client in std
zig/http.zig at 7cf2cbb33ef34c1d211135f56d30fe23b6cacd42 · ziglang/zig. General-purpose programming language and toolchain for maintaini...
github.com
/zig#http
0 5624 0
New
AstonJ
Backend>Questions

Psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
#macos/rails/postgresql
1 5553 1
New
AnfaengerAlex
Android>Questions

Unresolved Reference to android in build.gradle.kts – Beginner Issue
Hello, I’m a beginner in Android development and I’m facing an issue with my project setup. In my build.gradle.kts file, I have the foll...
#binding
0 7460 2
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 4275 15
New
Back to top

Sponsor Spotlight

EEF

Supporting innovation across the BEAM ecosystem.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap