Machine Learning in Elixir: error thrown in code snippet (page 44)

Pragmatic Bookshelf PragProg Customers
,
1

Running the code snippet of Chapter 2 on page 44 of version B1.0, I get the following error:

Evaluation process terminated - an exception was raised:
** (FunctionClauseError) no function clause matching in String.graphemes/1
(elixir 1.15.2) lib/string.ex:1934: String.graphemes(:ok)
(benchee 1.1.0) lib/benchee/utility/erlang_version.ex:100: Benchee.Utility.ErlangVersion.parse_erlang_version/1
(benchee 1.1.0) lib/benchee/utility/erlang_version.ex:84: Benchee.Utility.ErlangVersion.includes_fixes_from?/2
(benchee 1.1.0) lib/benchee/benchmark/repeated_measurement.ex:60: Benchee.Benchmark.RepeatedMeasurement.determine_resolution_adjustment/2
(benchee 1.1.0) lib/benchee/benchmark/repeated_measurement.ex:42: Benchee.Benchmark.RepeatedMeasurement.determine_n_times/5
(benchee 1.1.0) lib/benchee/benchmark/runner.ex:226: Benchee.Benchmark.Runner.measure_runtimes/4
(benchee 1.1.0) lib/benchee/benchmark/runner.ex:102: Benchee.Benchmark.Runner.measure_scenario/2
(elixir 1.15.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2

I run the code on an Apple Silicon M2 and I use Elixir 1.15.4 and Erlang/OTP 26.
I have not found any solution to this issue yet.

2

Update:

In the following snippet, each of the Pure Elixir and EXLA based softmax functions seem to work (not sure if I created the tensor correctly though).

The Benchee part still doesn’t work (gives the same error as reported above).

key = Nx.Random.key(1702)
{tensor, _new_key} = Nx.Random.uniform(key, shape: {1, 1_000_000})
Softmax.softmax(tensor)
apply(EXLA.jit(&Softmax.softmax/1), [tensor])

#Benchee.run(
#  %{
#    "JIT with EXLA" => fn -> apply(EXLA.jit(&Softmax.softmax/1), [tensor]) end,
#    "Regular Elixir" => fn -> Softmax.softmax(tensor) end
#  },
#  time: 10
#)
3

I had the same issue with Benchee - I saw someone mention in the elixir forum that Benchee was crashing the LiveBook cells a while back. I’m also on Apple Silicon - LiveBook v0.9.3, Elixir v1.14.2.

I was curious so:
{exla_time, exla_result} = :timer.tc(fn →
apply(EXLA.jit(&Softmax.softmax/1), [tensor])
end)

{standard_time, standard_result} = :timer.tc(fn →
Softmax.softmax(tensor)
end)

IO.puts “exla is #{standard_time/exla_time} times faster”
result (one run):
exla is 66.25130208333333 times faster

On multiple runs its between 450 and 500 times faster.

2 Likes
4

Thank you for the feedback and the alternative solution! Indeed, it seems to be an issue with Benchee.

In my local experiments, I get between 250-400 times faster.

5

Update:
By visiting the Elixir forum, I discovered that with switching to Benchee from the master branch on GitHub, the problem is solved.

Hence, I modified initial setup cell to:

Mix.install([
  {:nx, "~> 0.5"},
  {:exla, "~> 0.5"},
  {:benchee, github: "bencheeorg/benchee", override: true},
])

For the record, the improvement I noted was 337 times faster. I guess this is without using GPU, yet.

4 Likes
6

Just a note that I’ve updated the book to use the master branch as well :slight_smile: