AI>In The News

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

CommunityNews

CommunityNews

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

Today, Kog AI launches a tech preview of the Kog Inference Engine (KIE): 3,000 output tokens/s per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). This preview runs a 2B model, with support for large third-party MoE models coming next at similar speeds.

Read in full here:

https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/

View thread on forum
#llm
0 2 0

Where Next?

View thread on forum
llm
Home AI>In The News
#llm
0 2 0
Last post

Popular Ai topics Top

First poster: bot
AI>In The News

Nvidia Uses AI to Slash Bandwidth on Video Calls
NVIDIA Uses AI to Slash Bandwidth on Video Calls. NVIDIA Research has invented a way to use AI to dramatically reduce video call bandwid...
petapixel.com
#video#nvidia
1 966 0
New
First poster: jacobtriton
AI>In The News

Why AI is Harder Than We Think
Why AI is Harder Than We Think. Since its beginning in the 1950s, the field of artificial intelligence has cycled several times between...
arxiv.org
3 1471 2
New
First poster: bot
AI>In The News

Why cows may be hiding something but AI can spot it
bbc.co.uk
#spot
0 1004 0
New
First poster: bot
AI>In The News

AI Wrote and Performed a Jerry Seinfeld Routine
AI Wrote and Performed a Jerry Seinfeld Routine!. I used GPT-3 to write a Jerry Seinfeld stand-up routine about cats - and then used Dee...
youtube.com
#video
0 992 0
New
First poster: bot
AI>In The News

You can’t solve AI security problems with more AI
You can’t solve AI security problems with more AI. One of the most common proposed solutions to prompt injection attacks (where an AI la...
simonwillison.net
/security
0 1129 0
New
First poster: bot
AI>In The News

Adobe plays catch-up with Project Blink, an AI-powered video editor
AI video editor can recognize objects, people, and sounds, allowing editing via text.
arstechnica.com
#project#video#adobe
0 1271 0
New
First poster: bot
AI>In The News

AI and the Future of Pixel Art
AI and the Future of Pixel Art. Creative industries are undergoing a 0 to 1 moment. If you didn’t know, now you do. The impact that AI w...
pixelparmesan.com
#art
0 884 0
New
CommunityNews
AI>In The News

LLM Leaderboard - Comparison of over 100 AI models from OpenAI, Google, DeepSeek & others | Artificial Analysis
Comparison and ranking the performance of over 100 AI models (LLMs) across key metrics including intelligence, price, performance and spe...
artificialanalysis.ai
#google#artificial#openai#llm/deepseek
0 1250 0
New
First poster: jss
AI>In The News

Rob Pike (co-creator of Go) on GenAI
Fuck you people. Raping the planet, spending trillions on toxic, unrecyclable equipment while blowing up society, yet taking the time to ...
skyview.social
/go#claude#bluesky
5 268 5
New
First poster: apsori
AI>In The News

AI didn't delete your database, you did
Last week, a tweet went viral showing a guy claiming that a Cursor/Claude agent deleted his company’s production database. We watched fro...
idiallo.com
#database
0 157 2
New

Other popular topics Top

Devtalk
General Dev>Dev Chat

HELLO WORLD (Introductions thread!)
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
#community
482 7161 117
New
PragmaticBookshelf
General Dev>Learning Resources

A Common-Sense Guide to Data Structures and Algorithms, Second Edition
Algorithms and data structures are much more than abstract concepts. Mastering them enables you to write code that runs faster and more e...
pragprog.com
#pragprog/python/ruby#published-book/book-a-common-sense-guide-to-data-structures-and-algorithms-second-edition#math#algorithms/js
19 6022 5
New
dasdom
General Dev>Dev Chat

Standing Desks
No chair. I have a standing desk. This post was split into a dedicated thread from our thread about chairs :slight_smile:
#workspace#opinions
177 9886 77
New
AstonJ
General Dev>Dev Chat

Which language or framework do you want to learn next?
Curious to know which languages and frameworks you’re all thinking about learning next :upside_down_face: Perhaps if there’s enough peop...
#community#learning
243 6639 97
New
AstonJ
General Dev>Hardware

Custom keyboard keycaps
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
#hardware/keyboards#keycaps#mechanical-keyboards
15 11086 19
New
AstonJ
Backend>Questions

Erlang's not installing on macOS Big Sur "You are natively building Erlang/OTP for a later version of MacOSX than current version"
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
#macos/erlang#big-sur#asdf
10 6212 8
New
PragmaticBookshelf
Backend>Learning Resources

Concurrent Data Processing in Elixir
Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or...
pragprog.com
#pragprog/elixir#published-book/book-concurrent-data-processing-in-elixir
78 6059 24
New
AstonJ
Science/Tech>Health & Diet

David Sinclair's new Lifespan podcast
We’ve talked about his book briefly here but it is quickly becoming obsolete - so he’s decided to create a series of 7 podcasts, the firs...
#health#podcasts#bio-hackers#david-sinclair
87 6790 49
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Bruce Tate (Author) Interview and AMA!
Author Spotlight: Bruce Tate @redrapids Programming languages always emerge out of need, and if that’s not always true, they’re defin...
/elixir/ruby/phoenix/book-seven-more-languages-in-seven-weeks/book-seven-languages-in-seven-weeks#liveview/book-programming-phoenix-liveview
54 5678 23
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 4275 15
New
Back to top

Sponsor Spotlight

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

AI>In The News

AI In The News ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap