CommunityNews

CommunityNews

Deepseek - starting this week we'll open-source 5 repos

We’re a tiny team @deepseek-ai pushing our limits in AGI exploration.

Starting this week , Feb 24, 2025 we’ll open-source 5 repos – one daily drop – not because we’ve made grand claims, but simply as developers sharing our small-but-sincere progress with full transparency.

These are humble building blocks of our online service: documented, deployed and battle-tested in production. No vaporware, just sincere code that moved our tiny yet ambitious dream forward.

Why? Because every line shared becomes collective momentum that accelerates the journey. Daily unlocks begin soon. No ivory towers - just pure garage-energy and community-driven innovation :wrench:

Stay tuned – let’s geek out in the open together.

DeepSeek-Open-Infra

Hello, DeepSeek Open Infra!

202502 Open-Source Week

We’re a tiny team @deepseek-ai pushing our limits in AGI exploration.

Starting this week , Feb 24, 2025 we’ll open-source 5 repos – one daily drop – not because we’ve made grand claims,
but simply as developers sharing our small-but-sincere progress with full transparency.

These are humble building blocks of our online service: documented, deployed and battle-tested in production.
No vaporware, just sincere code that moved our tiny yet ambitious dream forward.

Why? Because every line shared becomes collective momentum that accelerates the journey.
Daily unlocks begin soon. No ivory towers - just pure garage-energy and community-driven innovation :wrench:

Stay tuned – let’s geek out in the open together.

Day 1 - FlashMLA

Efficient MLA Decoding Kernel for Hopper GPUs
Optimized for variable-length sequences, battle-tested in production

:link: FlashMLA GitHub Repo
:white_check_mark: BF16 support
:white_check_mark: Paged KV cache (block size 64)
:zap: Performance: 3000 GB/s memory-bound | BF16 580 TFLOPS compute-bound on H800

Day 2 - DeepEP

Excited to introduce DeepEP - the first open-source EP communication library for MoE model training and inference.

:link: DeepEP GitHub Repo
:white_check_mark: Efficient and optimized all-to-all communication
:white_check_mark: Both intranode and internode support with NVLink and RDMA
:white_check_mark: High-throughput kernels for training and inference prefilling
:white_check_mark: Low-latency kernels for inference decoding
:white_check_mark: Native FP8 dispatch support
:white_check_mark: Flexible GPU resource control for computation-communication overlapping

Day 3 - DeepGEMM

Introducing DeepGEMM - an FP8 GEMM library that supports both dense and MoE GEMMs, powering V3/R1 training and inference.

:link: DeepGEMM GitHub Repo
:zap: Up to 1350+ FP8 TFLOPS on Hopper GPUs
:white_check_mark: No heavy dependency, as clean as a tutorial
:white_check_mark: Fully Just-In-Time compiled
:white_check_mark: Core logic at ~300 lines - yet outperforms expert-tuned kernels across most matrix sizes
:white_check_mark: Supports dense layout and two MoE layouts

Day 4 - Optimized Parallelism Strategies

:white_check_mark: DualPipe - a bidirectional pipeline parallelism algorithm for computation-communication overlap in V3/R1 training.
:link: GitHub Repo

:white_check_mark: EPLB - an expert-parallel load balancer for V3/R1.
:link: GitHub Repo

:bar_chart: Analyze computation-communication overlap in V3/R1.
:link: GitHub Repo

Day 5 - 3FS, Thruster for All DeepSeek Data Access

Fire-Flyer File System (3FS) - a parallel file system that utilizes the full bandwidth of modern SSDs and RDMA networks.

:zap: 6.6 TiB/s aggregate read throughput in a 180-node cluster
:zap: 3.66 TiB/min throughput on GraySort benchmark in a 25-node cluster
:zap: 40+ GiB/s peak throughput per client node for KVCache lookup
:dna: Disaggregated architecture with strong consistency semantics
:white_check_mark: Training data preprocessing, dataset loading, checkpoint saving/reloading, embedding vector search & KVCache lookups for inference in V3/R1

:inbox_tray: 3FS → GitHub - deepseek-ai/3FS: A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
:fountain: Smallpond - data processing framework on 3FS → GitHub - deepseek-ai/smallpond: A lightweight data processing framework built on DuckDB and 3FS.

2024 AI Infrastructure Paper (SC24)

Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning

:page_facing_up: Paper Link
:page_facing_up: Arxiv Paper Link

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

Popular General Dev topics Top

First poster: bot
Typing is Hard Type Checking and Type Inference Common terms Completeness Soundness Decidability Hindley-Milner Type System Dependent t...
New
First poster: bot
A field guide to help you recognize achievement, spot A field guide to help you recognize achievement, spot bottlenecks, and debug your d...
New
New
CommunityNews
GitHub - livekit/livekit: Scalable, high-performance WebRTC SFU. SDKs in JavaScript, React, React Native, Flutter, Swift, Kotlin, Unity/C...
New
First poster: faust
Ruffle is a Flash Player emulator written in Rust. Ruffle runs natively on all modern operating systems as a standalone application, and ...
New
First poster: bot
Developing Godot Projects with Neovim. When I started using Godot Engine, what surprised me the most is the built-in Language Server Pro...
New
First poster: bot
Large Language Models like ChatGPT say The Darnedest Things. The Errors They MakeWhy We Need to Document Them, and What We Have Decided ...
New
First poster: bot
Rewrite it in Rust by ridiculousfish · Pull Request #9512 · fish-shell/fish-shell. (Sorry for the meme; also this is obligatory.) I thi...
New
First poster: bot
openai-python/chatml.md at main · openai/openai-python. The OpenAI Python library provides convenient access to the OpenAI API from appl...
New
New

Other popular topics Top

Devtalk
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
New
wolf4earth
@AstonJ prompted me to open this topic after I mentioned in the lockdown thread how I started to do a lot more for my fitness. https://f...
New
AstonJ
Curious to know which languages and frameworks you’re all thinking about learning next :upside_down_face: Perhaps if there’s enough peop...
New
New
AstonJ
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
New
New
AstonJ
Biggest jackpot ever apparently! :upside_down_face: I don’t (usually) gamble/play the lottery, but working on a program to predict the...
New
foxtrottwist
A few weeks ago I started using Warp a terminal written in rust. Though in it’s current state of development there are a few caveats (tab...
New
DevotionGeo
I have always used antique keyboards like Cherry MX 1800 or Cherry MX 8100 and almost always have modified the switches in some way, like...
New
PragmaticBookshelf
Author Spotlight: Tammy Coron @Paradox927 Gaming, and writing games in particular, is about passion, vision, experience, and immersio...
New