yamuna

yamuna

How to run Ollama deepseek-coder:6.7b-instruct-q4_K_M in Docker for CrewAI Agents?

Hi everyone,

I’m trying to run deepseek-coder:6.7b-instruct-q4_K_M in Docker using Ollama to create an LLM that will be used by CrewAI agents. However, I’m encountering errors while initializing the SQL Query Generator Agent.

Here’s the error log:

2025-03-12 06:17:59,984 - INFO - Creating SQL Query Generator Agent
2025-03-12 06:17:59,991 - ERROR - Failed to get supported params: argument of type ‘NoneType’ is not iterable
ERROR - LiteLLM call failed: litellm.BadRequestError: LLM Provider NOT provided.
Pass in the LLM provider you are trying to call. You passed model=base_url=‘http://ollama:11434’ model=‘deepseek-coder:6.7b-instruct-q4_K_M’ temperature=0.0

  • How should I properly specify the model provider?
  • Is there any special configuration required to make CrewAI recognize this model?
  • Has anyone successfully run this model in Docker with CrewAI?

Where Next?

Popular Backend topics Top

New
wolf4earth
At work we plan to replace a totally overkill Kafka instance with a combination of SNS and SQS. I don’t want to get into a discussion on ...
New
New
Fl4m3Ph03n1x
Background I am now trying Gradual type checking, as a consequence I am giving a shot to Gradient. As I see it, this is an alternative to...
New
AstonJ
If when trying to create (or recreate) your dev db with rails db:create you are getting: PG::ConnectionBad: connection to server on soc...
New
sona11
I’m having a difficulty. I want to modify an attribute’s data type from String to Array. { “id”: “trn:tarb:tradingpartner:uuid:00000...
New
sona11
I wrote this code to calculate Fibonacci numbers by specifying the size. The results are correct, however the one thing that concerns me ...
New
Fl4m3Ph03n1x
Background When trying to execute mix release on a Windows 11 machine for a Phoenix project I get the following error: * assembling mark...
New
harwind
I have a large SQL database with millions of records, and I’ve identified duplicate entries. What’s the most efficient way to find and re...
New
AstonJ
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
New

Other popular topics Top

PragmaticBookshelf
Write Elixir tests that you can be proud of. Dive into Elixir’s test philosophy and gain mastery over the terminology and concepts that u...
New
brentjanderson
Bought the Moonlander mechanical keyboard. Cherry Brown MX switches. Arms and wrists have been hurting enough that it’s time I did someth...
New
AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
AstonJ
We have a thread about the keyboards we have, but what about nice keyboards we come across that we want? If you have seen any that look n...
New
AstonJ
Biggest jackpot ever apparently! :upside_down_face: I don’t (usually) gamble/play the lottery, but working on a program to predict the...
New
PragmaticBookshelf
Author Spotlight Jamis Buck @jamis This month, we have the pleasure of spotlighting author Jamis Buck, who has written Mazes for Prog...
New
New
First poster: AstonJ
Jan | Rethink the Computer. Jan turns your computer into an AI machine by running LLMs locally on your computer. It’s a privacy-focus, l...
New
AstonJ
If you’re getting errors like this: psql: error: connection to server on socket “/tmp/.s.PGSQL.5432” failed: No such file or directory ...
New
PragmaticBookshelf
A concise guide to MySQL 9 database administration, covering fundamental concepts, techniques, and best practices. Neil Smyth MySQL...
New