Backend>Blogs/Talks

Inserting One Billion Rows in SQLite Under A Minute

CommunityNews

CommunityNews

Inserting One Billion Rows in SQLite Under A Minute

Recently, I ran into a situation where I needed a test database with lots of rows and needed it fast. So I did what any programmer would do: wrote a Python script to generate the DB. Unfortunately, it was slow. Really slow. So I did what any programmer would do: went down the rabbit hole of learning more about SQLite, Python, and eventually Rust… in my quest to get a 1B row database under a minute. This blog post is a summary of this fun and educational exercise.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
/python/rust#sqlite
0 844 0

Popular Backend topics Top

tomekzawada
Backend>Blogs/Talks

Live video streaming from iOS devices made simple with Elixir & Membrane Framework
Greetings from Membrane Framework team! Check out our case study based on our latest projects at Software Mansion. https://blog.swmansi...
blog.swmansion.com
/elixir/erlang#video-streaming#multimedia/membrane#swmansion
7 971 1
New
wolf4earth
Backend>Blogs/Talks

How we split Plug.SSL to control SSL redirection
dev.betterdoc.org
#https
5 1212 2
New
First poster: wolf4earth
Backend>Blogs/Talks

Understanding Partial Moves in Rust
Understanding Partial Moves in Rust. Partial moves are an interesting but often misunderstood feature of Rust. However, with the right ...
whileydave.com
/rust
8 1295 4
New
First poster: bot
Backend>Blogs/Talks

An Introduction to Crystal
One of my favourite programming languages in the last few years has been Crystal. While the language has not yet reached its 1.0 version,...
blog.oisinaylward.me
/crystal
0 1002 0
New
First poster: bot
Backend>Blogs/Talks

Rails Best Practices (video)
Rails Best Practices I. Today I share some of my favorite practices applicable to Ruby on Rails (and to web development on small teams g...
youtube.com
/rails#video
0 1002 0
New
AstonJ
Backend>Blogs/Talks

Euruko Keynote: Beyond Ruby 3.0 (Yukihiro Matsumoto)
Includes talk about concurrency and performance topics:
youtube.com
/ruby
62 1971 23
New
elbrujohalcon
Backend>Blogs/Talks

Spot the Difference
Erlang is famous for its introspecting powers. You can get a lot of information about the processes running in your nodes without any ext...
medium.com
/elixir/erlang#blog-post/gleam#efene
35 1346 6
New
First poster: bot
Backend>Blogs/Talks

BEAM all the things! ClojErl, an implementation of Clojure on the Erlang Virtual Machine
Our blog has had a long standing interest in novel uses of the BEAM, or Erlang virtual machine, as shown by the many articles we have pub...
notamonadtutorial.com
/clojure/erlang#beam#clojerl
2 1218 0
New
First poster: AstonJ
Backend>Blogs/Talks

Ruby Structs
Ruby’s Struct is one of several powerful core classes which is often overlooked and under utilized compared to the more popular Hash clas...
alchemists.io
/ruby
20 1456 13
New
tonyxrandall
Backend>Blogs/Talks

How to leverage functional programming in Kotlin to write better, cleaner code
As DoorDash transitioned from Python monolith to Kotlin microservices, our engineering team was presented with a lot of opportunities to ...
doordash.engineering
/kotlin#article#functional-programming
5 2705 1
New

Other popular topics Top

wolf4earth
Science/Tech>Health & Diet

How do you keep fit and healthy?
@AstonJ prompted me to open this topic after I mentioned in the lockdown thread how I started to do a lot more for my fitness. https://f...
#fitness#heath
193 4376 82
New
axelson
Cross Platform>Libraries/Tools

Obsidian – a cross platform app to help you create a knowledge base with Markdown files
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
#apps#macos#windows#linux/markdown#obsidian
38 3905 19
New
AstonJ
General Dev>Code Editors

Onivim 2 Code Editor
Thanks to @foxtrottwist’s and @Tomas’s posts in this thread: Poll: Which code editor do you use? I bought Onivim! :nerd_face: https://on...
#code-editors
88 5189 32
New
AstonJ
General Dev>Hardware

Planck vs Preonic vs Subatomic (Keyboards)
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
/keyboards#mechanical-keyboards#ortholinear#planck#preonic
105 14240 47
New
Margaret
General Dev>Content Creators

What tech topics do you think will (or should) be the focus of 2021?
Hello content creators! Happy new year. What tech topics do you think will be the focus of 2021? My vote for one topic is ethics in tech...
#general
110 3782 43
New
dimitarvp
macOS>Chat

My thoughts on macOS vs Linux
Small essay with thoughts on macOS vs. Linux: I know @Exadra37 is just waiting around the corner to scream at me “I TOLD YOU SO!!!” but I...
#macos#linux
166 7570 69
New
AstonJ
General Dev>Dev Chat

Languages Without Garbage Collection
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
#garbage-collection
21 4610 7
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Jamis Buck (Author) Interview and AMA!
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
#author-spotlight/ruby/book-the-ray-tracer-challenge/book-mazes-for-programmers
21 5463 9
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 10391 28
New
AstonJ
macOS>Chat

How to block any website on Mac using Little Snitch
If you want a quick and easy way to block any website on your Mac using Little Snitch simply… File > New Rule: And select Deny, O...
#macos#how-to#littlesnitch
5 7706 3
New
Back to top

Latest in Python

Python Portal ❯

Latest in Rust

Rust Portal ❯

Backend>Blogs/Talks

Backend Blogs/Talks ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap