Was the System One model Jev built to work with my choose-your-own-adventure
calculus model of programming systems? The fit is perfect! In this post, I run a couple
of experiments to evaluate accuracy and cost of Jev, using an experimental setup from my
recent paper, to be presented at SPLASH as part of Onward! Papers.
Read in full here:
The cost experiment would be more informative if it separates repeated state size from how many typed questions you evaluate per request. If the adventure trace is resent on every decision, deterministic removal of obsolete branches/history should cut input cost without changing the choice/score schema — the key check is whether compressed and full state produce the same decisions. At ~$0.042/MTok input and $0 output, savings should report almost linearly against removed input tokens.
