Update:
By visiting the Elixir forum, I discovered that with switching to Benchee
from the master branch on GitHub, the problem is solved.
Hence, I modified initial setup cell to:
Mix.install([
{:nx, "~> 0.5"},
{:exla, "~> 0.5"},
{:benchee, github: "bencheeorg/benchee", override: true},
])
For the record, the improvement I noted was 337 times faster. I guess this is without using GPU, yet.