AI>In The News

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

CommunityNews

CommunityNews

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

Today, Kog AI launches a tech preview of the Kog Inference Engine (KIE): 3,000 output tokens/s per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). This preview runs a 2B model, with support for large third-party MoE models coming next at similar speeds.

Read in full here:

https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/

View thread on forum
#llm
0 2 0

Where Next?

View thread on forum
llm
Home AI>In The News
#llm
0 2 0
Last post

Popular Ai topics Top

First poster: bot
AI>In The News

BlackBerry announces "industry first" AI-powered unified endpoint security platform
The new suite is composed of four products that cover endpoint protection, endpoint detection and response, mobile threat defense, and us...
techrepublic.com
/security#industry
0 1185 0
New
First poster: CommunityNews
AI>In The News

Europe seeks to limit use of AI in society
The use of facial recognition for surveillance, or algorithms that manipulate human behaviour, will be banned under proposed EU regulatio...
bbc.co.uk
0 1262 0
New
First poster: bot
AI>In The News

NVIDIA Canvas - create backgrounds with the help of AI
Use AI to turn simple brushstrokes into realistic landscape images. Create backgrounds quickly, or speed up your concept exploration so y...
nvidia.com
#nvidia
2 1085 0
New
CommunityNews
AI>In The News

The AI software that could turn you in to a music star
Artificial intelligence is now smart enough to write tracks that earn streaming service royalties.
bbc.co.uk
#music
4 1158 1
New
First poster: bot
AI>In The News

AI acquires the power to manipulate fusion, but wait, it’s actually good news
A research group has taught AI to magnetically wrangle a high-powered stream of plasma used for fusion research — but wait! Put away your...
techcrunch.com
#news
0 961 0
New
First poster: bot
AI>In The News

AI Wrote and Performed a Jerry Seinfeld Routine
AI Wrote and Performed a Jerry Seinfeld Routine!. I used GPT-3 to write a Jerry Seinfeld stand-up routine about cats - and then used Dee...
youtube.com
#video
0 992 0
New
First poster: bot
AI>In The News

AI and the Future of Pixel Art
AI and the Future of Pixel Art. Creative industries are undergoing a 0 to 1 moment. If you didn’t know, now you do. The impact that AI w...
pixelparmesan.com
#art
0 884 0
New
First poster: AstonJ
AI>In The News

AI: Where in the Loop Should Humans Go?
SRE Fred Hebert provides you with a list of questions to ask about potential AI solutions, including where humans should be involved.
honeycomb.io
/elixir/erlang/go
5 774 3
New
CommunityNews
AI>In The News

Tailwind has laid off 75% of their team
But the reality is that 75% of the people on our engineering team lost their jobs here yesterday because of the brutal impact AI has had ...
github.com
#tailwindcss#documentation#llm#llms
0 12 0
New
First poster: alvinkatojr
AI>In The News

Reasonix — DeepSeek-native AI coding agent
Open-source AI coding agent for your terminal. Engineered around DeepSeek’s prefix-cache. MCP first-class · plan mode · MIT.
esengine.github.io
#coding/deepseek#agent#native
1 1166 1
New

Other popular topics Top

PragmaticBookshelf
General Dev>Learning Resources

Seven More Languages in Seven Weeks
Learn from the award-winning programming series that inspired the Elixir language, and go on a step-by-step journey through the most impo...
pragprog.com
#pragprog/elixir/julia/lua#published-book#factor/elm#minikanren/idris/book-seven-more-languages-in-seven-weeks
4 5862 0
New
PragmaticBookshelf
Backend>Learning Resources

Testing Elixir
Write Elixir tests that you can be proud of. Dive into Elixir’s test philosophy and gain mastery over the terminology and concepts that u...
pragprog.com
#pragprog/elixir#published-book/book-testing-elixir
33 5004 8
New
brentjanderson
General Dev>Hardware

Moonlander Keyboard (Mechanical) (Ergonomic) (Split) (Ortholinear)
Bought the Moonlander mechanical keyboard. Cherry Brown MX switches. Arms and wrists have been hurting enough that it’s time I did someth...
#hardware/keyboards#moonlander#mechanical-keyboards#ortholinear#ergonomic
212 17779 90
New
Rainer
Community>Journals

Programming Erlang Book Club
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
/erlang/book-programming-erlang-2nd-edition#book-club
195 6815 95
New
AstonJ
Backend>Chat

How to install Ruby 3 with ASDF
In case anyone else is wondering why Ruby 3 doesn’t show when you do asdf list-all ruby :man_facepalming: do this first: asdf plugin-upd...
/ruby#asdf
11 5961 4
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Rebecca Skinner (Author) Interview and AMA!
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
#author-spotlight/haskell/book-effective-haskell
106 11719 28
New
PragmaticBookshelf
Community>In The Spotlight

Spotlight: Bruce Tate (Author) Interview and AMA!
Author Spotlight: Bruce Tate @redrapids Programming languages always emerge out of need, and if that’s not always true, they’re defin...
/elixir/ruby/phoenix/book-seven-more-languages-in-seven-weeks/book-seven-languages-in-seven-weeks#liveview/book-programming-phoenix-liveview
54 5678 23
New
PragmaticBookshelf
Backend>Learning Resources

C Brain Teasers
25 puzzles that will make you a better C programmer by challenging your knowledge of the language, and explaining the technical details o...
pragprog.com
#pragprog#published-book/c/book-c-brain-teasers
4 3873 2
New
AstonJ
AI>Chat

Post your DeepSeek results
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
/deepseek
15 4275 15
New
PragmaticBookshelf
Backend>Learning Resources

Risk-First Software Development, Second Edition
As digital systems increasingly run the world, mastery of the recurring patterns of software development risk is the key to fast and effe...
pragprog.com
#pragprog#published-book/book-risk-first-software-development-second-edition
12 4217 8
New
Back to top

Sponsor Spotlight

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

AI>In The News

AI In The News ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap