@seanmor5
Title: Machine Learning in elixir: Some problems with chapter 7 in Version B2.0
On 3 january 2024 I downloaded the Cats & Dogs file from the Kaggle site. I had to register and to subscribe to a contest to get it. All the images were named as x.jpg with x a number. They had to be renamed first and put in a train directory.
The renaming I did in a Linux terminal with:
For FILENAMe in *; do mv FILENAME cat.$FILENAME; done
For FILENAMe in *; do mv FILENAME dog.$FILENAME; done
Then running the livebook LearningToSee from chapter 7 I got errors at the cell from page 152:
" mlp_trained_model_state =".
1. Expected all shapes to match {x,96,96,3}, but got {x,96,96,4}
2. Could not decode binary of file
So there were some images wich did not match with the shape/channels.
To remove them I wrote the following module:
defmodule CleanCatsDogs do
def pipeline(paths) do
paths
|> Enum.map(fn x → x end)
end
def examine(x) do
{:ok, buffer} = File.read(x)
case StbImage.read_binary(buffer) do
{:ok, img} ->
process(img, x)
{:error, _} ->
IO.puts("Couldn't decode binary of #{x}")
File.rm(x)
end
end
def process(img, x) do
{_, _, c} = img.shape
if c != 3 do
IO.puts("#{c} -- #{x}")
File.rm(x)
end
end
end
and then executed:
train_path = Path.wildcard(“/home/piet/livebooks/train/*.jpg”)
train_line = CleanCatsDogs.pipeline(train_path)
Enum.map(train_line, fn x → CleanCatsDogs.examine(x) end)
Around 62 files were deleted.
Doing it in livebook is very easy. I’m very happy with this livebook examples.
Furthermore I got the warning at the for last cell “cnn_trained_model_state =”:
Axon.Optimizers.adam/1 is deprecated. Use Polaris.Optimizers.adam/1 instead
Well, simply replacing the name gave errors: FunctionClauseError etc.
I am not experienced enough to get a solution.
By the way: running this cell with 5 epochs on my desktop took me hours. So 100 epochs probably days. I stopped it. Didn’t get to the finish.