yamuna

yamuna

How to run Ollama deepseek-coder:6.7b-instruct-q4_K_M in Docker for CrewAI Agents?

Hi everyone,

I’m trying to run deepseek-coder:6.7b-instruct-q4_K_M in Docker using Ollama to create an LLM that will be used by CrewAI agents. However, I’m encountering errors while initializing the SQL Query Generator Agent.

Here’s the error log:

2025-03-12 06:17:59,984 - INFO - Creating SQL Query Generator Agent
2025-03-12 06:17:59,991 - ERROR - Failed to get supported params: argument of type ‘NoneType’ is not iterable
ERROR - LiteLLM call failed: litellm.BadRequestError: LLM Provider NOT provided.
Pass in the LLM provider you are trying to call. You passed model=base_url=‘http://ollama:11434’ model=‘deepseek-coder:6.7b-instruct-q4_K_M’ temperature=0.0

  • How should I properly specify the model provider?
  • Is there any special configuration required to make CrewAI recognize this model?
  • Has anyone successfully run this model in Docker with CrewAI?

Where Next?

Popular Backend topics Top

AstonJ
Just done a fresh install of macOS Big Sur and on installing Erlang I am getting: asdf install erlang 23.1.2 Configure failed. checking ...
New
Jsdr3398
Are there any databases that require no setup (can be shipped in a small zip together with the project)?
New
s2k
I have this code in a file that’s used to … render templates. require 'erb' require 'ostruct' MISSING_CONFIG_MARKER = :config_key_and_v...
New
Fl4m3Ph03n1x
Background I am trying to find a cheap and easy way to create New Types in Elixir, and Records seem to be just what I would need. Probl...
New
AstonJ
If when trying to create (or recreate) your dev db with rails db:create you are getting: PG::ConnectionBad: connection to server on soc...
New
Fl4m3Ph03n1x
Background I am moving towards defined data structures in my application, and I find that TypedStruct is quite useful. Questions Howeve...
New
Fl4m3Ph03n1x
Background I have a module that uses TypedStruct to create structs. This is the code: defmodule Shared.Data.Authorization do @moduledo...
New
Sumityadav
Hello all, I am new to learning Java Programming and want to learn java from scratch. I was writing a Java program to get the first and l...
New
yamuna
Hi everyone, I’m trying to run deepseek-coder:6.7b-instruct-q4_K_M in Docker using Ollama to create an LLM that will be used by CrewAI a...
New
Shiny
Hey community, this is my first post here so I will try to be as concise as possible and I appreciate any feedback. I’ve been writing Ro...
New

Other popular topics Top

Devtalk
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
New
AstonJ
If it’s a mechanical keyboard, which switches do you have? Would you recommend it? Why? What will your next keyboard be? Pics always w...
New
DevotionGeo
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
New
AstonJ
Thanks to @foxtrottwist’s and @Tomas’s posts in this thread: Poll: Which code editor do you use? I bought Onivim! :nerd_face: https://on...
New
Exadra37
I am asking for any distro that only has the bare-bones to be able to get a shell in the server and then just install the packages as we ...
New
AstonJ
In case anyone else is wondering why Ruby 3 doesn’t show when you do asdf list-all ruby :man_facepalming: do this first: asdf plugin-upd...
New
AstonJ
We’ve talked about his book briefly here but it is quickly becoming obsolete - so he’s decided to create a series of 7 podcasts, the firs...
New
PragmaticBookshelf
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
New
New
PragmaticBookshelf
Use advanced functional programming principles, practical Domain-Driven Design techniques, and production-ready Elixir code to build scal...
New