Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

First poster: bot
Backend>Blogs/Talks

The beauty of Phoenix LiveView
dashbit.co
/phoenix#liveview
2 1481 0
New
AstonJ
Backend>Blogs/Talks

The Erlang Rationale, by Robert Virding (PDF)
This article was written by @rvirding …over a decade ago! Posting here in case anyone else finds it of interest and adding it to our Erla...
drive.google.com
/erlang#sticky#info
7 1717 2
New
ErlangSolutions
Backend>Blogs/Talks

OCaml on the BEAM
Interested in a blazingly fast type checker with 25 years of investment, delivered on the BEAM? Check out Caramel, an exciting project fr...
www2.erlang-solutions.com
/erlang/ocaml#beam/caramel
5 1959 2
New
paulanthonywilson
Backend>Blogs/Talks

Making friends and influencing nodes, with multicasting
Post on using UDP multicasting with Elixir to broadcast presence, and listen for peers, on a local network. I have found this approach us...
furlough.merecomplexities.com
/elixir#blog-post#otp/nerves
2 1412 0
New
First poster: bot
Backend>Blogs/Talks

Speed of Rust vs. C
The run-time speed and memory usage of programs written in Rust should about the same as of programs written in C, but overall programmin...
kornel.ski
/c/rust
0 1354 0
New
First poster: malloryerik
Backend>Blogs/Talks

What is the most underestimated programming language?
Everyone outside of tech has heard of JavaScript, Java, Python, Ruby and even .Net, but few if any have heard of F#. However, F# may be o...
medium.com
#f-sharp#programming
43 2291 14
New
First poster: bot
Backend>Blogs/Talks

PHP in 2021
Once a year, I look back at the recent developments in the PHP world, and also look forward to what’s to come. And just like in 2020 and ...
stitcher.io
/php
0 1390 0
New
First poster: bot
Backend>Blogs/Talks

Julia is the better language for extending Python
Just a small test with lists in cython. Considering echosystem, multithreading and ease of use, Julia is a clear winner here.
github.com
/julia/python
0 1445 0
New
CommunityNews
Backend>Blogs/Talks

Functional Programming in Go with Generics
Functional programming is an increasing popular programming paradigm with many languages building or already supporting it. Go already su...
ani.dev
/go#programming
4 1625 1
New
First poster: bot
Backend>Blogs/Talks

A Clojure processor that's almost a drop-in replacement for php-fpm behind Nginx
Too long have we hustled to deploy Clojure websites. Too long have we spun up one server instance per site. Too long have reminisced abou...
github.com
/clojure/php#nginx
0 1472 0
New

Other popular topics Top

AstonJ
General Dev>Hardware

Which keyboard do you have?
If it’s a mechanical keyboard, which switches do you have? Would you recommend it? Why? What will your next keyboard be? Pics always w...
#hardware/keyboards#sticky#mechanical-keyboards
144 9115 50
New
Exadra37
Linux>Questions

AMD or Intel for Programming with Linux as the OS?
I am thinking in building or buy a desktop computer for programing, both professionally and on my free time, and my choice of OS is Linux...
#mobile#android#web-development#linux#desktop-computer#mobile-development
36 6006 10
New
dasdom
General Dev>Dev Chat

Standing Desks
No chair. I have a standing desk. This post was split into a dedicated thread from our thread about chairs :slight_smile:
#workspace#opinions
177 9886 77
New
DevotionGeo
General Dev>Code Editors

Dendron: a personal knowledge management tool on top of VSCode
/vscode#visual-studio-code
30 8077 9
New
Help
Game Dev>Questions

Can I use Java to program a game for Nintendo switch?
I am trying to crate a game for the Nintendo switch, I wanted to use Java as I am comfortable with that programming language. Can you use...
/java#nintendo
8 4771 3
New
First poster: CommunityNews
AI>In The News

How to fix the eyes in AI-generated images
aidemos.info
0 4508 0
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Bruce Tate (Author) Interview and AMA!
Author Spotlight: Bruce Tate @redrapids Programming languages always emerge out of need, and if that’s not always true, they’re defin...
/elixir/ruby/phoenix/book-seven-more-languages-in-seven-weeks/book-seven-languages-in-seven-weeks#liveview/book-programming-phoenix-liveview
54 5678 23
New
PragmaticBookshelf
General Dev>Learning Resources

A Common-Sense Guide to Data Structures and Algorithms in Python, Volume 1
Big O Notation can make your code faster by orders of magnitude. Get the hands-on info you need to master data structures and algorithms ...
pragprog.com
#pragprog/python#published-book/book-a-common-sense-guide-to-data-structures-and-algorithms-in-python-volume-1
24 5988 11
New
CommunityNews
General Dev>In The News

X can’t stop spread of explicit, fake AI Taylor Swift images
Will Swifties’ war on AI fakes spark a deepfake porn reckoning?
arstechnica.com
/swift
0 8379 0
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 4275 15
New
Back to top

Sponsor Spotlight

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap