Great points everyone! (I wonder if we should split these into a dedicated thread about a new benchmarking system?)
Ok so the benchmarks I have in mind would be based on real-world apps, these would be split into things such as a user registration system, i.e something from the real-world. Each submission would then be measured and rated.
There would be two types that can be submitted - the first would need to follow a very strict spec, eg, must use Postgres. The second would be where the submission could choose any technology so long as it can be used for the intended purpose in a live environment. (So number of errors/failures would factor in scoring and too many would render the submission ‘unfit for purpose’, thus disqualified.)
The benchmarks would be at benchmarks.devtalk.com and of course, in true DT fashion, there would be some twists and other cool things to make them even better 
Does that sound interesting to you all? 