AI>In The News

AGENTS.md outperforms skills in our agent evals

CommunityNews

CommunityNews

AGENTS.md outperforms skills in our agent evals

A compressed 8KB docs index in AGENTS.md achieved 100% on Next.js 16 API evals. Skills maxed at 79%. Here’s what we learned and how to set it up.

Read in full here:

1 comment
#vercel#agent
0 1337 1

First Post!

mira_tokarev

mira_tokarev

That 8KB index result matches what I’ve seen: a small, curated map often beats a larger bag of snippets because the agent can actually keep it in working context. The trade-off is staleness—I’d treat AGENTS.md as a stable contract (entry points, invariants, test commands) and fetch volatile details on demand.

For evals, I’d also separate retrieval cost from task success and test after deliberate context truncation. Did the benchmark measure whether the index stayed correct as APIs changed, or only a fixed snapshot?

Post #2

Where Next?

View thread on forum
vercel
agent
Home AI>In The News
#vercel#agent
0 1337 1
Last post

Popular Ai topics Top

First poster: CommunityNews
AI>In The News

In New Math Proofs, Artificial Intelligence Plays to Win
A new computer program fashioned after artificial intelligence systems like AlphaGo has solved several open problems in combinatorics and...
quantamagazine.org
#math
0 1287 0
New
First poster: CommunityNews
AI>In The News

Nvidia R&D chief on how AI is improving chip design
Getting a glimpse into Nvidia’s R&D has become a regular feature of the spring GTC conference with Bill Dally, chief scientist and se...
hpcwire.com
#nvidia#design
0 1188 0
New
First poster: bot
AI>In The News

Hyundai announces $400M AI, robotics institute powered by Boston Dynamics
When Hyundai acquired Boston Dynamics at the end of 2020, there were plenty of open questions. Chief among them was why we should assume ...
techcrunch.com
#robotics
0 929 0
New
First poster: DevotionGeo
AI>In The News

Fake Joe Rogan interviews fake Steve Jobs in an AI-powered podcast
Voice synthesis PR stunt calls upon the dead to help sell an AI product.
arstechnica.com
#jobs
2 913 3
New
First poster: joni
AI>In The News

My experience trying to write human-sounding articles using Claude AI
My experience trying to write original, full-length human-sounding articles using Claude AI. You can use AI tools like Claude to help yo...
idratherbewriting.com
#claude
0 1714 2
New
First poster: mercyf
AI>In The News

AI video just took a startling leap in realism. Are we doomed?
Google’s Veo 3 delivers AI videos of realistic people with sound and music. We put it to the test.
arstechnica.com
#video#veo
10 695 8
New
CommunityNews
AI>In The News

Cursor 1.0 - The AI Code Editor
Cursor 1.0 brings BugBot for code review, a first look at memories, one-click MCP setup, Jupyter support and general availability of Back...
cursor.com
#code#changelog#cursor
0 1040 0
New
CommunityNews
AI>In The News

AI Agent Benchmarks are Broken
Benchmarks are foundational to evaluating the strengths and limitations of AI systems, guiding both research and industry development.
ddkang.substack.com
0 1281 0
New
CommunityNews
AI>In The News

Defeating Nondeterminism in LLM Inference
Reproducibility is a bedrock of scientific progress. However, it’s remarkably difficult to get reproducible results out of large language...
thinkingmachines.ai
#llm
0 1270 0
New
CommunityNews
AI>In The News

The Gay Jailbreak - novel technique discovered against ChatGPT
:crescent_moon: ZetaLib - The only AI Library you need. Contribute to Exocija/ZetaLib development by creating an account on GitHub.
github.com
#chatgpt
0 1 0
New

Other popular topics Top

PragmaticBookshelf
Hybrid>Learning Resources

Programming Flutter
Instantly view the changes you make to an app with stateful hot reload, and define a declarative UI in the same language as the app logic...
pragprog.com
#pragprog/book-programming-flutter#published-book/flutter
11 4501 1
New
PragmaticBookshelf
General Dev>Learning Resources

Seven More Languages in Seven Weeks
Learn from the award-winning programming series that inspired the Elixir language, and go on a step-by-step journey through the most impo...
pragprog.com
#pragprog/elixir/julia/lua#published-book#factor/elm#minikanren/idris/book-seven-more-languages-in-seven-weeks
4 5862 0
New
DevotionGeo
General Dev>Code Editors

Dendron: a personal knowledge management tool on top of VSCode
/vscode#visual-studio-code
30 8077 9
New
AstonJ
Backend>Questions

Erlang's not installing on macOS Big Sur "You are natively building Erlang/OTP for a later version of MacOSX than current version"
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
#macos/erlang#big-sur#asdf
10 6212 8
New
PragmaticBookshelf
Backend>Learning Resources

Agile Web Development with Rails 7
Rails 7 completely redefines what it means to produce fantastic user experiences and provides a way to achieve all the benefits of single...
pragprog.com
#pragprog#web-development/ruby/rails#published-book/book-agile-web-development-with-rails-7
32 6600 9
New
AstonJ
Backend>Questions

Psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
#macos/rails/postgresql
1 5553 1
New
PragmaticBookshelf
Backend>Learning Resources

Agile Web Development with Rails 8
Get the comprehensive, insider information you need for Rails 8 with the new edition of this award-winning classic. Sam Ruby @rubys ...
pragprog.com
#pragprog#web-development/ruby/rails#published-book/book-agile-web-development-with-rails-8
12 7404 7
New
PragmaticBookshelf
Backend>Learning Resources

MySQL 9 Essentials
A concise guide to MySQL 9 database administration, covering fundamental concepts, techniques, and best practices. Neil Smyth MySQL...
pragprog.com
#pragprog#published-book/mysql/book-mysql-9-essentials
2 4162 0
New
PragmaticBookshelf
Backend>Learning Resources

Advanced Functional Programming with Elixir
Use advanced functional programming principles, practical Domain-Driven Design techniques, and production-ready Elixir code to build scal...
joekoski.com
#pragprog/elixir#published-book#functional-programming/book-advanced-functional-programming-with-elixir
43 4989 22
New
PragmaticBookshelf
Backend>Learning Resources

Risk-First Software Development, Second Edition
As digital systems increasingly run the world, mastery of the recurring patterns of software development risk is the key to fast and effe...
pragprog.com
#pragprog#published-book/book-risk-first-software-development-second-edition
12 4217 8
New
Back to top

Sponsor Spotlight

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

AI>In The News

AI In The News ❯

Latest on Devtalk

Devtalk ❯

We ❤️ helpful members!

We reward our most helpful members via our MOTM scheme - by giving away a whopping 25 books per year!

Member of the Month

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

Categories:

Sub Categories:

Popular Portals

Devtalk Sponsors

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Erlang Solutions

We build trusted fault-tolerant systems that scale to billions of users.

Honeybadger

Real-time error tracking, performance insights, and observability for devs.

EEF

Supporting innovation across the BEAM ecosystem.

coders51

We build reliable cloud platforms for business-critical systems.

Manning Publications

Develop your skills with books, videos, and courses.

AppSignal

Catch errors, track performance, monitor hosts and more.

WyeWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that move you from confusion to "Aha, now I get it!"

Linux New Media

Publishing Linux Magazine, ADMIN, and MakerSpace for 25+ years.

View all our sponsors ❯

We're in Beta

About us Mission Statement See our Roadmap