AI>In The News

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

CommunityNews

CommunityNews

Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)

Today, Kog AI launches a tech preview of the Kog Inference Engine (KIE): 3,000 output tokens/s per request on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). This preview runs a 2B model, with support for large third-party MoE models coming next at similar speeds.

Read in full here:

https://blog.kog.ai/real-time-llm-inference-on-standard-gpus-3-000-tokens-s-per-request/

View thread on forum
#llm
0 2 0

Where Next?

View thread on forum
llm
Home AI>In The News
#llm
0 2 0
Last post

Popular Ai topics Top

First poster: bot
AI>In The News

Combating Anti-Blackness in the AI Community
In response to a national and international awakening on the issues of anti-Blackness and systemic discrimination, we have penned this pi...
devinguillory.com
#community/diversity
0 1449 1
New
First poster: CommunityNews
AI>In The News

AI Can Generate Convincing Text–and Anyone Can Use It
SOME OF THE most dazzling recent advances in artificial intelligence have come thanks to resources only available at big tech companies, ...
wired.com
0 1576 0
New
First poster: bot
AI>In The News

Unveiling our new Quantum AI campus
Within the decade, Google aims to build a useful, error-corrected quantum computer. This will accelerate solutions for some of the world’...
blog.google
#google#quantum#tech-giants
0 1159 0
New
First poster: CommunityNews
AI>In The News

Nvidia R&D chief on how AI is improving chip design
Getting a glimpse into Nvidia’s R&D has become a regular feature of the spring GTC conference with Bill Dally, chief scientist and se...
hpcwire.com
#nvidia#design
0 1188 0
New
First poster: CommunityNews
AI>In The News

How to fix the eyes in AI-generated images
aidemos.info
0 4508 0
New
First poster: mercyf
AI>In The News

AI video just took a startling leap in realism. Are we doomed?
Google’s Veo 3 delivers AI videos of realistic people with sound and music. We put it to the test.
arstechnica.com
#video#veo
10 695 8
New
CommunityNews
AI>In The News

Claude Code is My Computer | Peter Steinberger
I run Claude Code with --dangerously-skip-permissions flag, giving it full system access. Let me show you a new way of approaching comput...
steipete.me
#code
0 914 0
New
First poster: AstonJ
AI>In The News

AI Changes Everything
From fear to optimism: why I am convinced AI is worth embracing.
lucumr.pocoo.org
4 854 5
New
CommunityNews
AI>In The News

Claude-code - native LSP support
Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing rout...
github.com
#code#changelog#claude
0 1 0
New
First poster: jss
AI>In The News

Rob Pike (co-creator of Go) on GenAI
Fuck you people. Raping the planet, spending trillions on toxic, unrecyclable equipment while blowing up society, yet taking the time to ...
skyview.social
/go#claude#bluesky
5 268 5
New

Other popular topics Top

Devtalk
General Dev>Dev Chat

What dev-related stuff have you been up to?
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
#community
1063 23582 394
New
PragmaticBookshelf
Game Dev>Learning Resources

Hands-on Rust: Effective Learning through 2D Game Development and Play
Rust is an exciting new programming language combining the power of C with memory safety, fearless concurrency, and productivity boosters...
pragprog.com
#pragprog/rust#published-book/book-hands-on-rust
117 10879 30
New
DevotionGeo
General Dev>Code Editors

Dendron: a personal knowledge management tool on top of VSCode
/vscode#visual-studio-code
30 8077 9
New
AstonJ
General Dev>Hardware

Planck vs Preonic vs Subatomic (Keyboards)
I ended up cancelling my Moonlander order as I think it’s just going to be a bit too bulky for me. I think the Planck and the Preonic (o...
/keyboards#mechanical-keyboards#ortholinear#planck#preonic
105 17596 47
New
Exadra37
Linux>Chat

RancherOS is in end of life
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
#linux#rancheros
10 6358 6
New
Margaret
General Dev>Dev Chat

PragProg’s Medium Posts
Hello everyone! This thread is to tell you about what authors from The Pragmatic Bookshelf are writing on Medium.
#pragprog#blog-post
1147 29994 760
New
foxtrottwist
General Dev>Dev Chat

Warp—The blazingly fast, Rust-based terminal
A few weeks ago I started using Warp a terminal written in rust. Though in it’s current state of development there are a few caveats (tab...
/rust#terminal
52 6785 22
New
mafinar
Backend>Chat

Data Structures and Algorithms with Elixir
This is going to be a long an frequently posted thread. While talking to a friend of mine who has taken data structure and algorithm cou...
/elixir#algorithms#data-structures
108 11869 31
New
hilfordjames
Windows>Chat

Taskbar Overflow Menu (NOT System Tray Overflow)
There appears to have been an update that has changed the terminology for what has previously been known as the Taskbar Overflow - this h...
#taskbar-overflow-win-11
3 3715 2
New
PragmaticBookshelf
Backend>Learning Resources

Engineering Elixir Applications
Develop, deploy, and debug BEAM applications using BEAMOps: a new paradigm that focuses on scalability, fault tolerance, and owning each ...
pragprog.com
#pragprog/elixir#published-book/book-engineering-elixir-applications
40 7136 21
New
Back to top

Sponsor Spotlight

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

AI>In The News

AI In The News ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap