pieteeken

pieteeken

Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

@seanmor5

Title: Machine Learning in elixir: Some problems with chapter 7 in Version B2.0

On 3 january 2024 I downloaded the Cats & Dogs file from the Kaggle site. I had to register and to subscribe to a contest to get it. All the images were named as x.jpg with x a number. They had to be renamed first and put in a train directory.

The renaming I did in a Linux terminal with:
For FILENAMe in *; do mv FILENAME cat.$FILENAME; done
For FILENAMe in *; do mv FILENAME dog.$FILENAME; done

Then running the livebook LearningToSee from chapter 7 I got errors at the cell from page 152:
" mlp_trained_model_state =".
1. Expected all shapes to match {x,96,96,3}, but got {x,96,96,4}
2. Could not decode binary of file

So there were some images wich did not match with the shape/channels.
To remove them I wrote the following module:


defmodule CleanCatsDogs do
def pipeline(paths) do
paths
|> Enum.map(fn x → x end)
end

def examine(x) do
{:ok, buffer} = File.read(x)

case StbImage.read_binary(buffer) do
  {:ok, img} ->
    process(img, x)

  {:error, _} ->
    IO.puts("Couldn't decode binary of #{x}")
    File.rm(x)
end

end

def process(img, x) do
{_, _, c} = img.shape

if c != 3 do
  IO.puts("#{c} -- #{x}")
  File.rm(x)
end

end
end


and then executed:


train_path = Path.wildcard(“/home/piet/livebooks/train/*.jpg”)
train_line = CleanCatsDogs.pipeline(train_path)
Enum.map(train_line, fn x → CleanCatsDogs.examine(x) end)


Around 62 files were deleted.
Doing it in livebook is very easy. I’m very happy with this livebook examples.

Furthermore I got the warning at the for last cell “cnn_trained_model_state =”:
Axon.Optimizers.adam/1 is deprecated. Use Polaris.Optimizers.adam/1 instead

Well, simply replacing the name gave errors: FunctionClauseError etc.
I am not experienced enough to get a solution.

By the way: running this cell with 5 epochs on my desktop took me hours. So 100 epochs probably days. I stopped it. Didn’t get to the finish.

First Post!

pieteeken

pieteeken

@seanmor5

Polaris problem solved.
See Hexdocs → Axon.Loop
Should be: Polaris.Optimizers.adam(learning_rate: 1.0e-3)

I lied: executing “improve training” took 2 hours with a non-cuda video card

Where Next?

Popular Pragmatic Bookshelf topics Top

New
iPaul
page 37 ANTLRInputStream input = new ANTLRInputStream(is); as of ANTLR 4 .8 should be: CharStream stream = CharStreams.fromStream(i...
New
jeffmcompsci
Title: Design and Build Great Web APIs - typo “https://company-atk.herokuapp.com/2258ie4t68jv” (page 19, third bullet in URL list) Typo:...
New
ianwillie
Hello Brian, I have some problems with running the code in your book. I like the style of the book very much and I have learnt a lot as...
New
mikecargal
Title: Hands-On Rust (Chapter 11: prefab) Just played a couple of amulet-less games. With a bit of debugging, I believe that your can_p...
New
herminiotorres
Hi! I know not the intentions behind this narrative when called, on page XI: mount() |> handle_event() |> render() but the correc...
New
adamwoolhether
When trying to generate the protobuf .go file, I receive this error: Unknown flag: --go_opt libprotoc 3.12.3 MacOS 11.3.1 Googling ...
New
jonmac
The allprojects block listed on page 245 produces the following error when syncing gradle: “org.gradle.api.GradleScriptException: A prob...
New
creminology
Skimming ahead, much of the following is explained in Chapter 3, but new readers (like me!) will hit a roadblock in Chapter 2 with their ...
New
a.zampa
@mfazio23 I’m following the indications of the book and arriver ad chapter 10, but the app cannot be compiled due to an error in the Bas...
New

Other popular topics Top

Devtalk
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
New
brentjanderson
Bought the Moonlander mechanical keyboard. Cherry Brown MX switches. Arms and wrists have been hurting enough that it’s time I did someth...
New
Rainer
My first contact with Erlang was about 2 years ago when I used RabbitMQ, which is written in Erlang, for my job. This made me curious and...
New
AstonJ
poll poll Be sure to check out @Dusty’s article posted here: An Introduction to Alternative Keyboard Layouts It’s one of the best write-...
New
AstonJ
Continuing the discussion from Thinking about learning Crystal, let’s discuss - I was wondering which languages don’t GC - maybe we can c...
New
PragmaticBookshelf
Author Spotlight Rebecca Skinner @RebeccaSkinner Welcome to our latest author spotlight, where we sit down with Rebecca Skinner, auth...
New
husaindevelop
Inside our android webview app, we are trying to paste the copied content from another app eg (notes) using navigator.clipboard.readtext ...
New
PragmaticBookshelf
Leverage your existing Rails codebase to build iOS and Android apps with Hotwire Native – no Swift or Kotlin experience necessary. J...
New
PragmaticBookshelf
Explore the power of Ash Framework by modeling and building the domain for a real-world web application. Rebecca Le @sevenseacat and ...
New
PragmaticBookshelf
Escape callback hell and ship fast, clean code that reads as smoothly as it runs. Squash bugs and stamp out memory leaks with an intuitiv...
New

Sub Categories: