pieteeken

pieteeken

Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

@seanmor5

Title: Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

On 3 january 2024 I downloaded the Cats & Dogs file from the Kaggle site. I had to register and to subscribe to a contest to get it. All the images were named as x.jpg with x a number. They had to be renamed first and put in a train directory.

The renaming I did in a Linux terminal with:
For FILENAMe in *; do mv FILENAME cat.$FILENAME; done
For FILENAMe in *; do mv FILENAME dog.$FILENAME; done

Then running the livebook LearningToSee from chapter 7 I got errors at the cell from page 152:
" mlp_trained_model_state =".
1. Expected all shapes to match {x,96,96,3}, but got {x,96,96,4}
2. Could not decode binary of file

So there were some images wich did not match with the shape/channels.
To remove them I wrote the following module:


defmodule CleanCatsDogs do
def pipeline(paths) do
paths
|> Enum.map(fn x → x end)
end

def examine(x) do
{:ok, buffer} = File.read(x)

case StbImage.read_binary(buffer) do
  {:ok, img} ->
    process(img, x)

  {:error, _} ->
    IO.puts("Couldn't decode binary of #{x}")
    File.rm(x)
end

end

def process(img, x) do
{_, _, c} = img.shape

if c != 3 do
  IO.puts("#{c} -- #{x}")
  File.rm(x)
end

end
end


and then executed:


train_path = Path.wildcard(“/home/piet/livebooks/train/*.jpg”)
train_line = CleanCatsDogs.pipeline(train_path)
Enum.map(train_line, fn x → CleanCatsDogs.examine(x) end)


Around 62 files were deleted.
Doing it in livebook is very easy. I’m very happy with this livebook examples.

Furthermore I got the warning at the for last cell “cnn_trained_model_state =”:
Axon.Optimizers.adam/1 is deprecated. Use Polaris.Optimizers.adam/1 instead

Well, simply replacing the name gave errors: FunctionClauseError etc.
I am not experienced enough to get a solution.

By the way: running this cell with 5 epochs on my desktop took me hours. So 100 epochs probably days. I stopped it. Didn’t get to the finish.

First Post!

pieteeken

pieteeken

@seanmor5

Polaris problem solved.
See Hexdocs → Axon.Loop
Should be: Polaris.Optimizers.adam(learning_rate: 1.0e-3)

I lied: executing “improve training” took 2 hours with a non-cuda video card

Where Next?

Popular Pragmatic Bookshelf topics Top

Mmm
Hi, build fails on: bracket-lib = “~0.8.1” when running on Mac Mini M1 Rust version 1.5.0: Compiling winit v0.22.2 error[E0308]: mi...
New
conradwt
First, the code resources: Page 237: rumbl_umbrella/apps/rumbl/mix.exs Note: That this file is missing. Page 238: rumbl_umbrella/app...
New
jeremyhuiskamp
Title: Web Development with Clojure, Third Edition, vB17.0 (p9) The create table guestbook syntax suggested doesn’t seem to be accepted ...
New
swlaschin
The book has the same “Problem space/Solution space” diagram on page 18 as is on page 17. The correct Problem/Solution space diagrams ar...
New
jskubick
I’m under the impression that when the reader gets to page 136 (“View Data with the Database Inspector”), the code SHOULD be able to buil...
New
nicoatridge
Hi, I have just acquired Michael Fazio’s “Kotlin and Android Development” to learn about game programming for Android. I have a game in p...
New
New
mert
AWDWR 7, page 152, page 153: Hello everyone, I’m a little bit lost on the hotwire part. I didn’t fully understand it. On page 152 @rub...
New
Henrai
Hi, I’m working on the Chapter 8 of the book. After I add add the point_offset, I’m still able to see acne: In the image above, I re...
New
jwandekoken
Book: Programming Phoenix LiveView, page 142 (157/378), file lib/pento_web/live/product_live/form_component.ex, in the function below: d...
New

Other popular topics Top

DevotionGeo
I know that -t flag is used along with -i flag for getting an interactive shell. But I cannot digest what the man page for docker run com...
New
Exadra37
Oh just spent so much time on this to discover now that RancherOS is in end of life but Rancher is refusing to mark the Github repo as su...
New
AstonJ
Biggest jackpot ever apparently! :upside_down_face: I don’t (usually) gamble/play the lottery, but working on a program to predict the...
New
mafinar
This is going to be a long an frequently posted thread. While talking to a friend of mine who has taken data structure and algorithm cou...
New
PragmaticBookshelf
Author Spotlight Mike Riley @mriley This month, we turn the spotlight on Mike Riley, author of Portable Python Projects. Mike’s book ...
New
New
husaindevelop
Inside our android webview app, we are trying to paste the copied content from another app eg (notes) using navigator.clipboard.readtext ...
New
AstonJ
This is a very quick guide, you just need to: Download LM Studio: https://lmstudio.ai/ Click on search Type DeepSeek, then select the o...
New
AstonJ
Curious what kind of results others are getting, I think actually prefer the 7B model to the 32B model, not only is it faster but the qua...
New
Margaret
Ask Me Anything with Mark Volkmann @mvolkmann On February 24 and 25, we are giving you a chance to ask questions of PragProg author M...
New

Sub Categories: