General Dev>In The News

LLM inference speed of light

CommunityNews

CommunityNews

LLM inference speed of light

LLM inference speed of light.
In the process of working on calm, a minimal from-scratch fast CUDA implementation of transformer-based language model inference, a critical consideration was establishing the speed of light for the inference process, and measuring the progress relative to that speed of light. In this post we’ll cover this theoretical limit and its implications.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

View thread on forum
#llm
0 284 0

Where Next?

View thread on forum
llm
Home General Dev>In The News
#llm
0 284 0
Last post

Popular General Dev topics Top

First poster: HenryCost
General Dev>In The News

I wired my tree with 500 LED lights and calculated their 3D coordinates
I wired my tree with 500 LED lights and calculated their 3D coordinates… If you support me on Patreon at any point in December 2020 I wi...
youtube.com
1 1491 2
New
First poster: bot
General Dev>In The News

20 patterns to watch for in your engineering team (pdf)
A field guide to help you recognize achievement, spot A field guide to help you recognize achievement, spot bottlenecks, and debug your d...
pluralsight.com
0 1172 0
New
First poster: jaisonv
General Dev>In The News

Google has a secret deal with FB called “Jedi Blue” that they knew was illegal
twitter.com
#google
42 1353 15
New
First poster: malloryerik
General Dev>In The News

Doom-emacs: An Emacs framework
GitHub - hlissner/doom-emacs: An Emacs framework for the stubborn martian hacker. An Emacs framework for the stubborn martian hacker - G...
github.com
/emacs#doom-emacs
55 3198 16
New
First poster: dimitarvp
General Dev>In The News

A career ending mistake
A career ending mistake — Bitfield Consulting. As software engineers, we’re constantly making detailed, elaborate plans for computers to...
bitfieldconsulting.com
#career
22 1374 8
New
First poster: bot
General Dev>In The News

The overengineered Solution to my Pigeon Problem
The overengineered Solution to my Pigeon Problem. TL;DR: I built a wifi-equipped water gun to shoot the pigeons on my balcony, controlle...
maxnagy.com
0 4512 0
New
First poster: cpgo
General Dev>In The News

8 Reasons to ditch Chrome and use Firefox
8 reasons to ditch Chrome and switch to Firefox. Chrome may dominate, but Firefox is a known name among browsers for a reason. Whether y...
pcworld.com
#chrome#firefox
73 1728 41
New
First poster: bot
General Dev>In The News

A more mature ChatGPT competitor? PaLM + RLHF - Pytorch (wip)
GitHub - lucidrains/PaLM-rlhf-pytorch: Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architectur...
github.com
#chatgpt
0 1245 0
New
First poster: joni
General Dev>In The News

My experience trying to write human-sounding articles using Claude AI
My experience trying to write original, full-length human-sounding articles using Claude AI. You can use AI tools like Claude to help yo...
idratherbewriting.com
0 1327 1
New
First poster: alvinkatojr
General Dev>In The News

Distributed Systems Programming Has Stalled
Over the last decade, we’ve seen great advancements in distributed systems, but the way we program them has seen few fundamental improvem...
shadaj.me
#programming
4 531 2
New

Other popular topics Top

ohm
Science/Tech>Tech Chat

Games! Which do you play?
Which, if any, games do you play? On what platform? I just bought (and completed) Minecraft Dungeons for my Nintendo Switch. Other than ...
#games
246 5882 101
New
wolf4earth
Science/Tech>Health & Diet

How do you keep fit and healthy?
@AstonJ prompted me to open this topic after I mentioned in the lockdown thread how I started to do a lot more for my fitness. https://f...
#fitness#heath
193 4728 82
New
AstonJ
General Dev>Code Editors

Poll: Which code editor do you use?
You might be thinking we should just ask who’s not using VSCode :joy: however there are some new additions in the space that might give V...
#community#polls/vim/emacs#code-editors/vscode#notepad/sublime-text#atom/textmate#codespaces#brackets/onivim#geany
121 5409 61
New
AstonJ
General Dev>Hardware

BIIP MT3 Extended 2048 Custom Keycap Set (Drop)
This looks like a stunning keycap set :orange_heart: A LEGENDARY KEYBOARD LIVES ON When you bought an Apple Macintosh computer in the e...
/keyboards#apple#keycaps#mechanical-keyboards
14 6365 7
New
Exadra37
Linux>Questions

What is the most minimalist Linux server distro?
I am asking for any distro that only has the bare-bones to be able to get a shell in the server and then just install the packages as we ...
#linux#servers
66 23964 24
New
PragmaticBookshelf
Backend>Learning Resources

Programming WebRTC
Use WebRTC to build web applications that stream media and data in real time directly from one user to another, all in the browser. ...
pragprog.com
#pragprog#published-book/js#webrtc/book-programming-webrtc
27 4743 6
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 10968 28
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Karl Stolley (Author) Interview and AMA!
Author Spotlight: Karl Stolley @karlstolley Logic! Rhetoric! Prag! Wow, what a combination. In this spotlight, we sit down with Karl ...
#author-spotlight/book-programming-webrtc
31 4135 16
New
AstonJ
AI>Chat

How to: Run DeepSeek on Mac, Windows, and Linux!
This is a very quick guide, you just need to: Download LM Studio: https://lmstudio.ai/ Click on search Type DeepSeek, then select the o...
#macs/deepseek#guides#lm-studio
14 7706 10
New
PragmaticBookshelf
Backend>Learning Resources

Simplicity
Fight complexity and reclaim the original spirit of agility by learning to simplify how you develop software. The result: a more humane a...
pragprog.com
#pragprog#published-book/book-simplicity
10 4288 8
New
Back to top

Sponsor Spotlight

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

General Dev>In The News

General Dev In The News ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap