pieteeken

pieteeken

Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

@seanmor5

Title: Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

On 3 january 2024 I downloaded the Cats & Dogs file from the Kaggle site. I had to register and to subscribe to a contest to get it. All the images were named as x.jpg with x a number. They had to be renamed first and put in a train directory.

The renaming I did in a Linux terminal with:
For FILENAMe in *; do mv FILENAME cat.$FILENAME; done
For FILENAMe in *; do mv FILENAME dog.$FILENAME; done

Then running the livebook LearningToSee from chapter 7 I got errors at the cell from page 152:
" mlp_trained_model_state =".
1. Expected all shapes to match {x,96,96,3}, but got {x,96,96,4}
2. Could not decode binary of file

So there were some images wich did not match with the shape/channels.
To remove them I wrote the following module:


defmodule CleanCatsDogs do
def pipeline(paths) do
paths
|> Enum.map(fn x → x end)
end

def examine(x) do
{:ok, buffer} = File.read(x)

case StbImage.read_binary(buffer) do
  {:ok, img} ->
    process(img, x)

  {:error, _} ->
    IO.puts("Couldn't decode binary of #{x}")
    File.rm(x)
end

end

def process(img, x) do
{_, _, c} = img.shape

if c != 3 do
  IO.puts("#{c} -- #{x}")
  File.rm(x)
end

end
end


and then executed:


train_path = Path.wildcard(“/home/piet/livebooks/train/*.jpg”)
train_line = CleanCatsDogs.pipeline(train_path)
Enum.map(train_line, fn x → CleanCatsDogs.examine(x) end)


Around 62 files were deleted.
Doing it in livebook is very easy. I’m very happy with this livebook examples.

Furthermore I got the warning at the for last cell “cnn_trained_model_state =”:
Axon.Optimizers.adam/1 is deprecated. Use Polaris.Optimizers.adam/1 instead

Well, simply replacing the name gave errors: FunctionClauseError etc.
I am not experienced enough to get a solution.

By the way: running this cell with 5 epochs on my desktop took me hours. So 100 epochs probably days. I stopped it. Didn’t get to the finish.

First Post!

pieteeken

pieteeken

@seanmor5

Polaris problem solved.
See Hexdocs → Axon.Loop
Should be: Polaris.Optimizers.adam(learning_rate: 1.0e-3)

I lied: executing “improve training” took 2 hours with a non-cuda video card

Where Next?

Popular Pragmatic Bookshelf topics Top

jon
Some minor things in the paper edition that says “3 2020” on the title page verso, not mentioned in the book’s errata online: p. 186 But...
New
jdufour
Hello! On page xix of the preface, it says there is a community forum "… for help if your’re stuck on one of the exercises in this book… ...
New
raul
Page 28: It implements io.ReaderAt on the store type. Sorry if it’s a dumb question but was the io.ReaderAt supposed to be io.ReadAt? ...
New
brunogirin
When installing Cards as an editable package, I get the following error: ERROR: File “setup.py” not found. Directory cannot be installe...
New
brunogirin
When running tox for the first time, I got the following error: ERROR: InterpreterNotFound: python3.10 I realised that I was running ...
New
taguniversalmachine
It seems the second code snippet is missing the code to set the current_user: current_user: Accounts.get_user_by_session_token(session["...
New
tkhobbes
After some hassle, I was able to finally run bin/setup, now I have started the rails server but I get this error message right when I vis...
New
Keton
When running the program in chapter 8, “Implementing Combat”, the printout Health before attack was never printed so I assumed something ...
New
mcpierce
@mfazio23 I’ve applied the changes from Chapter 5 of the book and everything builds correctly and runs. But, when I try to start a game,...
New
dachristenson
I just bought this book to learn about Android development, and I’m already running into a major issue in Ch. 1, p. 20: “Update activity...
New

Other popular topics Top

axelson
I’ve been really enjoying obsidian.md: It is very snappy (even though it is based on Electron). I love that it is all local by defaul...
New
siddhant3030
I’m thinking of buying a monitor that I can rotate to use as a vertical monitor? Also, I want to know if someone is using it for program...
New
AstonJ
Curious to know which languages and frameworks you’re all thinking about learning next :upside_down_face: Perhaps if there’s enough peop...
New
Rainer
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
New
AstonJ
There’s a whole world of custom keycaps out there that I didn’t know existed! Check out all of our Keycaps threads here: https://forum....
New
AstonJ
I’ve been hearing quite a lot of comments relating to the sound of a keyboard, with one of the most desirable of these called ‘thock’, he...
New
AstonJ
I have seen the keycaps I want - they are due for a group-buy this week but won’t be delivered until October next year!!! :rofl: The Ser...
New
Margaret
Hello everyone! This thread is to tell you about what authors from The Pragmatic Bookshelf are writing on Medium.
1147 28379 760
New
rustkas
Intensively researching Erlang books and additional resources on it, I have found that the topic of using Regular Expressions is either c...
New
AstonJ
Was just curious to see if any were around, found this one: I got 51/100: Not sure if it was meant to buy I am sure at times the b...
New

Sub Categories: