Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 1129 0

Where Next?

View thread on forum
python
rust
sqlite
Home Backend>Blogs/Talks
/python/rust#sqlite
0 1129 0
Last post

Popular Backend topics Top

CommunityNews
Backend>Blogs/Talks

Optimization - Making Rust Code Go Brrrr
aspenuwu.me
/go/rust#code
2 1429 1
New
First poster: bot
Backend>Blogs/Talks

Where (and how) you limit your concurrency in Go can matter
utcc.utoronto.ca
/go
0 1576 0
New
First poster: bot
Backend>Blogs/Talks

How NOT to Teach Recursion
We all know how to teach recursion. We’ve done it for decades. We pick some honored, time-tested examples—Fibonacci numbers and factorial...
parentheticallyspeaking.org
/elixir/erlang/gleam/lfe/caramel
1 1525 0
New
First poster: bot
Backend>Blogs/Talks

Revisiting a Video Chat Application with Phoenix and WebRTC
In a previous post we talked about implementing a simple video chat with WebRTC and Elixir. This update will touch on some of the API cha...
hashrocket.com
/phoenix#chat#video#webrtc
1 1956 0
New
AstonJ
Backend>Blogs/Talks

How to set-up a Ruby and Elixir dev environment on macOS (2021)
Just finished doing a clean install of macOS (which I highly recommend btw!) and have updated my macOS Ruby & Elixir/Erlang dev env s...
astonj.com
/elixir/erlang/ruby/phoenix/nodejs#asdf
3 1889 0
New
brainlid
Backend>Blogs/Talks

ThinkingElixir 052 - IOList and Postgres with Nathan Long
We take a deeper dive with Nathan Long into IOLists in Elixir. We cover what they are, how they work, the power they have when concatenat...
thinkingelixir.com
/elixir/phoenix#postgres#podcasts
5 1318 0
New
brainlid
Backend>Blogs/Talks

Copy to Clipboard with Phoenix LiveView
Jason Stiebs shows a couple ways for a LiveView to make it easy for users to click and copy an important value to their clipboard. He sho...
fly.io
/elixir/phoenix#blog-post#liveview
10 1131 2
New
fullstackplus
Backend>Blogs/Talks

Bulding apps data-first with Sinatra
The Ruby ecosystem is rich with tools that make us developers more productive at what we do. Both Rails and Sinatra have been used to bui...
fullstackplus.tech
/ruby/rails/sinatra#functional-programming
21 1381 10
New
brainlid
Backend>Blogs/Talks

Created my Personal AI Fitness Trainer in 2 Days
In a 2 day spike, I created my own Elixir-based AI Personal Fitness Trainer! The surprising part for me was how useful and helpful I foun...
fly.io
#ai/elixir#blog-post#fitness#chatgpt#workout
2 1120 0
New
GoulvenClech
Backend>Blogs/Talks

Directory structure with Elixir/Phoenix
Hi everyone :wave: I’m excited to share an article detailing how we have reorganized our Elixir/Phoenix project’s directory structure. W...
goulven-clech.dev
/elixir/phoenix#blog-post#folder-structure
9 1191 3
New

Other popular topics Top

PragmaticBookshelf
Backend>Learning Resources

Programming Machine Learning
Machine learning can be intimidating, with its reliance on math and algorithms that most programmers don't encounter in their regular wor...
pragprog.com
#pragprog#ai/python#published-book/book-programming-machine-learning#math#algorithms
6 5350 3
New
DevotionGeo
Backend>Chat

What is the reason behind Rust’s web framework, Rocket, not performing as well as expected in the Techempower benchmarks?
I know that these benchmarks might not be the exact picture of real-world scenario, but still I expect a Rust web framework performing a ...
#web-frameworks/rust
36 7463 11
New
PragmaticBookshelf
Game Dev>Learning Resources

Apple Game Frameworks and Technologies
Design and develop sophisticated 2D games that are as much fun to make as they are to play. From particle effects and pathfinding to soci...
pragprog.com
#pragprog#ios#game-dev#macos/swift#published-book#apple/book-apple-game-frameworks-and-technologies
30 7995 10
New
brentjanderson
General Dev>Hardware

Moonlander Keyboard (Mechanical) (Ergonomic) (Split) (Ortholinear)
Bought the Moonlander mechanical keyboard. Cherry Brown MX switches. Arms and wrists have been hurting enough that it’s time I did someth...
#hardware/keyboards#moonlander#mechanical-keyboards#ortholinear#ergonomic
212 17779 90
New
AstonJ
Backend>Questions

Erlang's not installing on macOS Big Sur "You are natively building Erlang/OTP for a later version of MacOSX than current version"
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
#macos/erlang#big-sur#asdf
10 6212 8
New
AstonJ
General Dev>Dev Chat

Languages Without Garbage Collection
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
#garbage-collection
21 5575 7
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 11719 28
New
husaindevelop
Android>Questions

Clipboard readtext not working in android webview
Inside our android webview app, we are trying to paste the copied content from another app eg (notes) using navigator.clipboard.readtext ...
#android#clipboard
1 5651 0
New
CommunityNews
General Dev>In The News

X can’t stop spread of explicit, fake AI Taylor Swift images
Will Swifties’ war on AI fakes spark a deepfake porn reckoning?
arstechnica.com
/swift
0 8379 0
New
PragmaticBookshelf
Backend>Learning Resources

Advanced Functional Programming with Elixir
Use advanced functional programming principles, practical Domain-Driven Design techniques, and production-ready Elixir code to build scal...
joekoski.com
#pragprog/elixir#published-book#functional-programming/book-advanced-functional-programming-with-elixir
43 4989 22
New
Back to top

Sponsor Spotlight

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap