Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

AstonJ
Backend>Blogs/Talks

The Erlang Rationale, by Robert Virding (PDF)
This article was written by @rvirding …over a decade ago! Posting here in case anyone else finds it of interest and adding it to our Erla...
drive.google.com
/erlang#sticky#info
7 1717 2
New
First poster: bot
Backend>Blogs/Talks

Tutorial – Write a Shell in C
It’s easy to view yourself as “not a real programmer.” There are programs out there that everyone uses, and it’s easy to put their develo...
brennan.io
/c
0 1699 0
New
First poster: bot
Backend>Blogs/Talks

Speed of Rust vs. C
The run-time speed and memory usage of programs written in Rust should about the same as of programs written in C, but overall programmin...
kornel.ski
/c/rust
0 1354 0
New
prajaut
Backend>Blogs/Talks

Golang Error Handling blog post
Being a part of the tech industry, it would be good to share thoughts on specific technologies. Having surrounded by skilled and experie...
bacancytechnology.com
/go
7 1898 2
New
First poster: brennan
Backend>Blogs/Talks

Java Is Underhyped
The perspective of an ignorant computer science undergrad It’s likely that you read the title of this post and thought “what is this guy ...
jackson.sh
/java
8 1598 4
New
First poster: bot
Backend>Blogs/Talks

Julia is the better language for extending Python
Just a small test with lists in cython. Considering echosystem, multithreading and ease of use, Julia is a clear winner here.
github.com
/julia/python
0 1445 0
New
axelson
Backend>Blogs/Talks

Hot Reloading: A Faster Way to Develop Modern Apps
I describe how we use Hot Reloading with Webpack to develop faster and show how to integrate Webpack 5, webpack-dev-server, and Phoenix f...
felt.com
/elixir#blog-post
9 1472 3
New
brainlid
Backend>Blogs/Talks

Safe Ecto Migrations - Recipes and Resources
There is a new community resource available on writing “Safe Ecto Migrations”. When we get a migration wrong, it can lock up your product...
fly.io
/elixir#blog-post#postgres#database
4 1417 0
New
elbrujohalcon
Backend>Blogs/Talks

Inaka / TDD, Coverage and Why Testing Exceptions
Another week, another oldies-but-goldies post… This one about Test Driven Development.
tealfeed.com
/erlang#article#test#tdd
4 1421 0
New
brainlid
Backend>Blogs/Talks

ThinkingElixir 092 - Temple with Mitchell Hanberg
In episode 92 of Thinking Elixir, we talk with Mitchell Hanberg and learn about why he created the alternate Phoenix templating language ...
podcast.thinkingelixir.com
/elixir#podcasts#testing#browser/html#dev-news-source
4 1327 0
New

Other popular topics Top

PragmaticBookshelf
General Dev>Learning Resources

Seven More Languages in Seven Weeks
Learn from the award-winning programming series that inspired the Elixir language, and go on a step-by-step journey through the most impo...
pragprog.com
#pragprog/elixir/julia/lua#published-book#factor/elm#minikanren/idris/book-seven-more-languages-in-seven-weeks
4 5862 0
New
PragmaticBookshelf
Game Dev>Learning Resources

Apple Game Frameworks and Technologies
Design and develop sophisticated 2D games that are as much fun to make as they are to play. From particle effects and pathfinding to soci...
pragprog.com
#pragprog#ios#game-dev#macos/swift#published-book#apple/book-apple-game-frameworks-and-technologies
30 7995 10
New
PragmaticBookshelf
Android>Learning Resources

Kotlin and Android Development featuring Jetpack: Build Better, Safer Android Apps
Start building native Android apps the modern way in Kotlin with Jetpack's expansive set of tools, libraries, and best practices. Learn h...
pragprog.com
#pragprog#android#game-dev/kotlin#published-book/book-kotlin-and-android-development-featuring-jetpack
7 5084 1
New
AstonJ
Science/Tech>Health & Diet

David Sinclair's new Lifespan podcast
We’ve talked about his book briefly here but it is quickly becoming obsolete - so he’s decided to create a series of 7 podcasts, the firs...
#health#podcasts#bio-hackers#david-sinclair
87 6790 49
New
PragmaticBookshelf
Backend>Learning Resources

Agile Web Development with Rails 7
Rails 7 completely redefines what it means to produce fantastic user experiences and provides a way to achieve all the benefits of single...
pragprog.com
#pragprog#web-development/ruby/rails#published-book/book-agile-web-development-with-rails-7
32 6600 9
New
Help
Game Dev>Questions

Can I use Java to program a game for Nintendo switch?
I am trying to crate a game for the Nintendo switch, I wanted to use Java as I am comfortable with that programming language. Can you use...
/java#nintendo
8 4771 3
New
AstonJ
Backend>Questions

Psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
#macos/rails/postgresql
1 5553 1
New
PragmaticBookshelf
Backend>Learning Resources

The New and Improved Flask Mega-Tutorial
Overarching tutorial for Python beginner and intermediate developers that teaches web development with the Flask framework. Miguel Gr...
blog.miguelgrinberg.com
#pragprog/python/flask#published-book/book-the-new-and-improved-flask-mega-tutorial
1 3553 0
New
mindriot
General Dev>Reviews

Keyboard Review: UHK60V2 vs Defy vs Voyager vs Glove80 vs Svalboard
Ok, well here are some thoughts and opinions on some of the ergonomic keyboards I have, I guess like mini review of each that I use enoug...
/keyboards#uhk60v2#defy#voyager#glove80#svalboard
5 5681 7
New
PragmaticBookshelf
General Dev>Learning Resources

Write Better with Vale
Lint your docs like code: turn any style guide into enforceable rules with Vale and publish clear, consistent content every time. ...
pragprog.com
#pragprog#published-book#vale/book-write-better-with-vale
8 9973 5
New
Back to top

Sponsor Spotlight

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap