Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1027 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1027 0
Last post

Popular Backend topics Top

First poster: bot
Backend>Blogs/Talks

Where (and how) you limit your concurrency in Go can matter
utcc.utoronto.ca
/go
0 1425 0
New
AstonJ
Backend>Blogs/Talks

The Erlang Rationale, by Robert Virding (PDF)
This article was written by @rvirding …over a decade ago! Posting here in case anyone else finds it of interest and adding it to our Erla...
drive.google.com
/erlang#sticky#info
7 1538 2
New
CommunityNews
Backend>Blogs/Talks

Is Zig the Long Awaited C Replacement
Is Zig the Long Awaited C Replacement. Comparison with previous C contenders such as C++, D, Java, C#, Go, Rust and Swift https://erik...
erik-engheim.medium.com
/c/c-plus-plus/rust/swift/zig/v
4 2143 1
New
First poster: bot
Backend>Blogs/Talks

Revisiting a Video Chat Application with Phoenix and WebRTC
In a previous post we talked about implementing a simple video chat with WebRTC and Elixir. This update will touch on some of the API cha...
hashrocket.com
/phoenix#chat#video#webrtc
1 1821 0
New
First poster: bot
Backend>Blogs/Talks

Tutorial – Write a Shell in C
It’s easy to view yourself as “not a real programmer.” There are programs out there that everyone uses, and it’s easy to put their develo...
brennan.io
/c
0 1448 0
New
AstonJ
Backend>Blogs/Talks

Ruby Creator Yukihiro Matsumoto talks about Ruby 3 (interview)
Not had time to read it yet but this looks like a good interview… Our friend Yukihiro Matsumoto, creator of the Ruby programming langua...
evrone.com
/ruby#interiew
1 1542 0
New
First poster: bot
Backend>Blogs/Talks

Speed of Rust vs. C
The run-time speed and memory usage of programs written in Rust should about the same as of programs written in C, but overall programmin...
kornel.ski
/c/rust
0 1217 0
New
First poster: OvermindDL1
Backend>Blogs/Talks

The Gleam Programming Language and its Creator Louis Pilfold
youtube.com
/gleam#programming
16 1537 4
New
First poster: bot
Backend>Blogs/Talks

From Julia to Rust
I’ve been more serious about learning Rust recently, after dragging on with passive learning for a while. My first real programming langu...
miguelraz.github.io
/julia/rust
2 1487 0
New
tonyxrandall
Backend>Blogs/Talks

Migrating From Python to Kotlin for Our Backend Services (DoorDash)
When DoorDash approached the limits of what our Django-based monolithic codebase could support, we needed to design a new stack that woul...
doordash.engineering
/kotlin/python#blog-post#backend
3 3321 0
New

Other popular topics Top

DevotionGeo
Backend>Chat

What is the reason behind Rust’s web framework, Rocket, not performing as well as expected in the Techempower benchmarks?
I know that these benchmarks might not be the exact picture of real-world scenario, but still I expect a Rust web framework performing a ...
#web-frameworks/rust
36 7115 11
New
dasdom
General Dev>Dev Chat

Standing Desks
No chair. I have a standing desk. This post was split into a dedicated thread from our thread about chairs :slight_smile:
#workspace#opinions
177 9043 77
New
DevotionGeo
Backend>Questions

Can someone explain the -t option/flag in docker run command?
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
#docker
7 9055 2
New
DevotionGeo
Backend>Chat

Would you use Erlang now when there is Elixir?
Why, if your answer is yes?
/elixir/erlang
167 4700 52
New
First poster: bot
Backend>In The News

Bit – A modernized Git CLI written in Go
github.com
/go#git
1 5741 0
New
AstonJ
Backend>Questions

Erlang's not installing on macOS Big Sur "You are natively building Erlang/OTP for a later version of MacOSX than current version"
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
#macos/erlang#big-sur#asdf
10 5914 8
New
AstonJ
Backend>Chat

Rails console using 100% CPU in dev (fix)
If you are experiencing Rails console using 100% CPU on your dev machine, then updating your development and test gems might fix the issu...
/ruby/rails
3 3971 3
New
PragmaticBookshelf
Backend>Learning Resources

Programming WebRTC
Use WebRTC to build web applications that stream media and data in real time directly from one user to another, all in the browser. ...
pragprog.com
#pragprog#published-book/js#webrtc/book-programming-webrtc
27 4743 6
New
foxtrottwist
General Dev>Dev Chat

Warp—The blazingly fast, Rust-based terminal
A few weeks ago I started using Warp a terminal written in rust. Though in it’s current state of development there are a few caveats (tab...
/rust#terminal
52 5824 22
New
husaindevelop
Android>Questions

Clipboard readtext not working in android webview
Inside our android webview app, we are trying to paste the copied content from another app eg (notes) using navigator.clipboard.readtext ...
#android#clipboard
1 4678 0
New
Back to top

Sponsor Spotlight

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap