Machine Learning in Elixir: Comments on Chapter 12, Version B 3.0

Comments on Chapter 12, Version B 3.0

@seanmor5

All the Axon.Optimizers.adam(…) should be replaced by Polaris.Optimizers.adam( learning_rate: …)

Section Learning a Structured Latent

line 31: epsilon = Nx.random_normal(noise_shape) should be replaced by:

key = Nx.Random.key(42)
{epsilon, new_key} = Nx.Random.normal(key, shape: noise_shape)

line 98: decoder_params = decoder_init_fn.(Nx.random_uniform({64, 2}), init_state)
should be replaced by:

key = Nx.Random.key(100)
{decoder_params, _new_key} = decoder_init_fn.(Nx.Random.uniform(key, shape: {64, 2}),init_state)

In the last cell of this section there is an Error.
Line 10: |> Axon.Loop.run(train_data, %{}, compiler: EXLA, epochs: 10)
Don’t know how to handle it.

** (ArithmeticError) bad argument in arithmetic expression: {2} * 1
    :erlang.*({2}, 1)
    (elixir 1.15.7) lib/tuple.ex:167: Tuple.product/2
    (elixir 1.15.7) lib/tuple.ex:167: Tuple.product/2
    (axon 0.6.1) lib/axon/initializers.ex:603: Axon.Initializers.compute_fans/1
    (axon 0.6.1) lib/axon/initializers.ex:578: Axon.Initializers."__defn:variance_scaling_impl__"/3
    (axon 0.6.1) lib/axon/compiler.ex:1092: Axon.Compiler.init_param/6
    (elixir 1.15.7) lib/enum.ex:2510: Enum."-reduce/3-lists^foldl/2-0-"/3
    #cell:k4cy577ik7plpwyh:10: (file)

Section Generating with GANs

line 39: g_params = generator_init_fn.(Nx.random_normal({64, 128}), init_state)
should be replaced by:

key = Nx.Random.key(42)
g_params = generator_init_fn.(Nx.Random.normal(key, shape: {64, 128}), init_state)

line 58 +70: latent = Nx.random_normal({batch_size, 128})
should be replaced by:

key = Nx.Random.key(100)
latent = Nx.Random.normal(key, shape: {batch_size, 128})    

line 143: latent = Nx.random_normal({3, 128})
should be replaced by:

key = Nx.Random.key(42)
latent = Nx.Random.normal(key, shape: {3, 128})

Then the last cell.
There is a warning: Axon.Loop.handle/3 is deprecated. handle/4 is deprecated, use handle_event/4 instead.
That line is: |> Axon.Loop.handle(:epoch_completed, &GAN.display_sample(&1, generator_fn))
It should be:

|> Axon.Loop.handle_event(:epoch_completed, &GAN.display_sample(&1, generator_fn))

There is also an error I don’t know to solve. The same as in the section before
Line ten of the last cell: |> Axon.Loop.run(train_data, %{}, compiler: EXLA, epochs: 10)

15:46:28.070 [debug] Forwarding options: [compiler: EXLA] to JIT compiler
** (ArithmeticError) bad argument in arithmetic expression: {2} * 1
    :erlang.*({2}, 1)
    (elixir 1.15.7) lib/tuple.ex:167: Tuple.product/2
    (elixir 1.15.7) lib/tuple.ex:167: Tuple.product/2
    (axon 0.6.1) lib/axon/initializers.ex:603: Axon.Initializers.compute_fans/1
    (axon 0.6.1) lib/axon/initializers.ex:578: Axon.Initializers."__defn:variance_scaling_impl__"/3
    (axon 0.6.1) lib/axon/compiler.ex:1092: Axon.Compiler.init_param/6
    (elixir 1.15.7) lib/enum.ex:2510: Enum."-reduce/3-lists^foldl/2-0-"/3
    #cell:3qkqd37egg5g6jsz:10: (file)