Hey @seanmor5,

I’m having great training accuracy but poor evaluation accuracy for the example in Chapter 1 when following the code in the book.

Epoch: 9, Batch: 450, accuracy: 0.9750040 loss: 0.2519934

Batch: 0, accuracy: 0.0000000
%{
  0 => %{
    "accuracy" => #Nx.Tensor<
      f32
      0.0
    >
  }
}

After some trouble, I realised that the book’s code diverges from the accompanying livebooks from PragProg. Here’s the code in question.

Book, Page 16:

...
train_categories =
  train_df["species"]
  |> Explorer.Series.cast(:category)

y_train =
  train_categories
  |> Nx.stack(axis: -1)
  |> Nx.equal(Nx.iota({1, 3}, axis: -1))

x_test = Nx.stack(test_df[feature_columns], axis: 1)

test_categories =
  test_df["species"]
  |> Explorer.Series.cast(:category)

y_test =
  test_categories
  |> Nx.stack(axis: -1)
  |> Nx.equal(Nx.iota({1, 3}, axis: -1))

Accompanying Livebook:

...
y_train =
  train_df
  |> DF.pull(label_column)
  |> Explorer.Series.to_list()
  |> Enum.map(fn
    "Iris-setosa" -> 0
    "Iris-versicolor" -> 1
    "Iris-virginica" -> 2
  end)
  |> Nx.tensor(type: :u8)
  |> Nx.new_axis(-1)
  |> Nx.equal(Nx.iota({1, 3}, axis: -1))
...
y_test =
  test_df
  |> DF.pull(label_column)
  |> Explorer.Series.to_list()
  |> Enum.map(fn
    "Iris-setosa" -> 0
    "Iris-versicolor" -> 1
    "Iris-virginica" -> 2
  end)
  |> Nx.tensor(type: :u8)
  |> Nx.new_axis(-1)
  |> Nx.equal(Nx.iota({1, 3}, axis: -1))

Seems like it had to do with the ordering of the categories and how it maps when doing the one-hot encoding.

There’s a thread started on Elixirforum where @grossvogel more succintly explains what’s happening & with some alternative code.