Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 939 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 939 0
Last post

Popular Backend topics Top

CommunityNews
Backend>Blogs/Talks

Optimization - Making Rust Code Go Brrrr
aspenuwu.me
/go/rust#code
2 1207 1
New
First poster: iPaul
Backend>Blogs/Talks

The Future of Clojure
thoughtworks.com
/clojure#podcasts
3 1460 2
New
First poster: bot
Backend>Blogs/Talks

Revisiting a Video Chat Application with Phoenix and WebRTC
In a previous post we talked about implementing a simple video chat with WebRTC and Elixir. This update will touch on some of the API cha...
hashrocket.com
/phoenix#chat#video#webrtc
1 1698 0
New
paulanthonywilson
Backend>Blogs/Talks

Blog on using binary WebSocket messages with Elixir & Phoenix
So you’re enjoying using WebSockets with Elixir’s Phoenix Framework, and you want to send some binary messages. Maybe it’s an audio clip,...
furlough.merecomplexities.com
/elixir/phoenix#blog-post#websockets
6 1387 2
New
First poster: dimitarvp
Backend>Blogs/Talks

I rewrote my Rust keyboard firmware in Zig: consistency, mastery, and fun
I’ve spent the last year building keyboards, which has included writing firmware for a variety custom circuit boards. I initially wrote ...
kevinlynagh.com
/rust/zig
3 1684 2
New
paulanthonywilson
Backend>Blogs/Talks

Making friends and influencing nodes, with multicasting
Post on using UDP multicasting with Elixir to broadcast presence, and listen for peers, on a local network. I have found this approach us...
furlough.merecomplexities.com
/elixir#blog-post#otp/nerves
2 1198 0
New
First poster: bot
Backend>Blogs/Talks

Running Lisp in Production
At Grammarly, the foundation of our business, our core grammar engine, is written in Common Lisp. It currently processes more than a thou...
grammarly.com
/lisp#production
1 1103 0
New
elbrujohalcon
Backend>Blogs/Talks

Inaka / TDD, Coverage and Why Testing Exceptions
Another week, another oldies-but-goldies post… This one about Test Driven Development.
tealfeed.com
/erlang#article#test#tdd
4 1142 0
New
First poster: AstonJ
Backend>Blogs/Talks

Ruby Structs
Ruby’s Struct is one of several powerful core classes which is often overlooked and under utilized compared to the more popular Hash clas...
alchemists.io
/ruby
20 1573 13
New
fullstackplus
Backend>Blogs/Talks

Bulding apps data-first with Sinatra
The Ruby ecosystem is rich with tools that make us developers more productive at what we do. Both Rails and Sinatra have been used to bui...
fullstackplus.tech
/ruby/rails/sinatra#functional-programming
21 1010 10
New

Other popular topics Top

AstonJ
General Dev>Hardware

Which keyboard do you have?
If it’s a mechanical keyboard, which switches do you have? Would you recommend it? Why? What will your next keyboard be? Pics always w...
#hardware/keyboards#sticky#mechanical-keyboards
144 8502 50
New
DevotionGeo
Backend>Chat

What is the reason behind Rust’s web framework, Rocket, not performing as well as expected in the Techempower benchmarks?
I know that these benchmarks might not be the exact picture of real-world scenario, but still I expect a Rust web framework performing a ...
#web-frameworks/rust
36 6636 11
New
AstonJ
General Dev>Hardware

Planck vs Preonic vs Subatomic (Keyboards)
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
/keyboards#mechanical-keyboards#ortholinear#planck#preonic
105 14440 47
New
Margaret
General Dev>Content Creators

What tech topics do you think will (or should) be the focus of 2021?
Hello content creators! Happy new year. What tech topics do you think will be the focus of 2021? My vote for one topic is ethics in tech...
#general
110 3900 43
New
PragmaticBookshelf
Backend>Learning Resources

Programming Phoenix LiveView
Build highly interactive applications without ever leaving Elixir, the way the experts do. Let LiveView take care of performance, scalabi...
pragprog.com
#pragprog/elixir/phoenix#published-book/book-programming-phoenix-liveview
61 4304 16
New
AstonJ
Data Science

Can AI/ML predict a lottery win?
Biggest jackpot ever apparently! :upside_down_face: I don’t (usually) gamble/play the lottery, but working on a program to predict the...
#ai#machine-learning
19 3178 10
New
PragmaticBookshelf
Backend>Learning Resources

Effective Haskell
Build efficient applications that exploit the unique benefits of a pure functional language, learning from an engineer who uses Haskell t...
pragprog.com
#pragprog/haskell#published-book/book-effective-haskell
15 5398 1
New
First poster: bot
General Dev>In The News

The overengineered Solution to my Pigeon Problem
The overengineered Solution to my Pigeon Problem. TL;DR: I built a wifi-equipped water gun to shoot the pigeons on my balcony, controlle...
maxnagy.com
0 4367 0
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Erin Dees (Author) Interview and AMA!
Author Spotlight Erin Dees @undees Welcome to our new author spotlight! We had the pleasure of chatting with Erin Dees, co-author of ...
#author-spotlight/ruby/rails/book-effective-testing-with-rspec-3/book-seven-more-languages-in-seven-weeks/book-cucumber-recipes#rspec
24 3704 11
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 1138 15
New
Back to top

Sponsor Spotlight

AppSignal

Catch errors, track performance, monitor hosts and more.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap