@pieteeken I changed it as follows, fixed it for me:
opts = Keyword.validate!(opts, [:defn_options, sequence_length: 64, batch_size: 16])
{defn_opts, compile_opts} = Keyword.pop!(opts, :defn_options)
opts = [defn_options: defn_opts, compile: compile_opts]