Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

@seanmor5

Title: Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

On 3 january 2024 I downloaded the Cats & Dogs file from the Kaggle site. I had to register and to subscribe to a contest to get it. All the images were named as x.jpg with x a number. They had to be renamed first and put in a train directory.

The renaming I did in a Linux terminal with:
For FILENAMe in *; do mv FILENAME cat.$FILENAME; done
For FILENAMe in *; do mv FILENAME dog.$FILENAME; done

Then running the livebook LearningToSee from chapter 7 I got errors at the cell from page 152:
" mlp_trained_model_state =".
1. Expected all shapes to match {x,96,96,3}, but got {x,96,96,4}
2. Could not decode binary of file

So there were some images wich did not match with the shape/channels.
To remove them I wrote the following module:

defmodule CleanCatsDogs do
def pipeline(paths) do
paths
|> Enum.map(fn x → x end)
end

def examine(x) do
{:ok, buffer} = File.read(x)

case StbImage.read_binary(buffer) do
  {:ok, img} ->
    process(img, x)

  {:error, _} ->
    IO.puts("Couldn't decode binary of #{x}")
    File.rm(x)
end

end

def process(img, x) do
{_, _, c} = img.shape

if c != 3 do
  IO.puts("#{c} -- #{x}")
  File.rm(x)
end

end
end

and then executed:

train_path = Path.wildcard(“/home/piet/livebooks/train/*.jpg”)
train_line = CleanCatsDogs.pipeline(train_path)
Enum.map(train_line, fn x → CleanCatsDogs.examine(x) end)

Around 62 files were deleted.
Doing it in livebook is very easy. I’m very happy with this livebook examples.

Furthermore I got the warning at the for last cell “cnn_trained_model_state =”:
Axon.Optimizers.adam/1 is deprecated. Use Polaris.Optimizers.adam/1 instead

Well, simply replacing the name gave errors: FunctionClauseError etc.
I am not experienced enough to get a solution.

By the way: running this cell with 5 epochs on my desktop took me hours. So 100 epochs probably days. I stopped it. Didn’t get to the finish.