createos

createos

How are you handling the full deployment lifecycle for AI workloads in production?

Curious how other teams are approaching this.

Building an AI app used to mean picking a model and writing product logic. Now it means also picking a hosting provider, wiring up a monitoring tool, and at some point figuring out billing. Three separate systems, each with their own failure modes, each needing maintenance.

The pattern I keep seeing: teams ship something that works in staging, then spend the next month firefighting the infrastructure around it. A monitoring alert lags the actual incident by 10+ minutes. The billing integration breaks when usage spikes. The hosting layer that worked for a prototype can not handle real traffic.

Some specific questions for anyone running AI workloads in production:

  • Are you managing hosting, monitoring, and billing as separate systems or have you consolidated them?

  • If separate, how much engineering time per week goes into keeping those integrations running vs. building the actual product?

  • Have you looked at managed execution layers as an alternative to self-building this stack?

We ran into this problem ourselves while building CreateOS (createos.sh), which ended up being our answer to it. But I am more interested in how others are solving it, or whether the problem is even the same across different team sizes.

Where Next?

Popular General Dev topics Top

AstonJ
Split from a thread about Serverless architectures. Serverless/cloud from Wikipedia: Serverless computing is a cloud computing execut...
New
KnowledgeIsPower
I had written an article about Migrate a K3S cluster storage from Rook to OpenEBS, with Velero Please give comment or feedback, thanks!
New
KnowledgeIsPower
I had written an article about Using mcrouter and memcached as caching layer for Thanos Store Please give comment or feedback, thanks!
New
New
akinihsan
I am trying to use following kubernetes ingress service to host dashboard in the server. it doesnt host the administration dashboard. How...
New
akinihsan
Hello Do you use static code analysiss tool on your ci cd pipeline? Which one do u use and what are the benefits for you
New
tasaraskam
Hey everyone, I’m a DevOps engineer and I noticed that most developers either don’t know DevOps or hate setting up CI/CD and Docker. So...
New
runmyjob
Hi, We’re building EU-based CI runners with load-based billing at RunMyJob and I’d love to benchmark them against real-world CI setups. ...
New
Amzy31
CharleX WebOS CharleX WebOS makes the web feel like a desktop computer. It allows you to use windows, applications, storage, notes, an...
New
almokhtar
A standard deploy is binary: the new release passes its health check, and all traffic moves to it at once. That’s fine until the thing th...
New

Other popular topics Top

Devtalk
Reading something? Working on something? Planning something? Changing jobs even!? If you’re up for sharing, please let us know what you’...
1063 24245 395
New
PragmaticBookshelf
Free and open source software is the default choice for the technologies that run our world, and it’s built and maintained by people like...
New
PragmaticBookshelf
Ruby, Io, Prolog, Scala, Erlang, Clojure, Haskell. With Seven Languages in Seven Weeks, by Bruce A. Tate, you’ll go beyond the syntax—and...
New
Exadra37
Please tell us what is your preferred monitor setup for programming(not gaming) and why you have chosen it. Does your monitor have eye p...
New
New
AstonJ
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
New
Exadra37
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
New
First poster: bot
zig/http.zig at 7cf2cbb33ef34c1d211135f56d30fe23b6cacd42 · ziglang/zig. General-purpose programming language and toolchain for maintaini...
New
PragmaticBookshelf
Escape callback hell and ship fast, clean code that reads as smoothly as it runs. Squash bugs and stamp out memory leaks with an intuitiv...
New
CommunityNews
The pro-Coup faction fabricated statements, misrepresented classified documents, and laundered baseless opinion as fact.
New