A Generalist Neural Algorithmic Learner.
The cornerstone of neural algorithmic reasoning is the ability to solve
algorithmic tasks, especially in a way that generalises out of distribution.
While recent years have seen a surge in methodological improvements in this
area, they mostly focused on building specialist models. Specialist models are
capable of learning to neurally execute either only one algorithm or a
collection of algorithms with identical control-flow backbone. Here, instead,
we focus on constructing a generalist neural algorithmic learner – a single
graph neural network processor capable of learning to execute a wide range of
algorithms, such as sorting, searching, dynamic programming, path-finding and
geometry. We leverage the CLRS benchmark to empirically show that, much like
recent successes in the domain of perception, generalist algorithmic learners
can be built by “incorporating” knowledge. That is, it is possible to
effectively learn algorithms in a multi-task manner, so long as we can learn to
execute them well in a single-task regime. Motivated by this, we present a
series of improvements to the input representation, training regime and
processor architecture over CLRS, improving average single-task performance by
over 20% from prior art. We then conduct a thorough ablation of multi-task
learners leveraging these improvements. Our results demonstrate a generalist
learner that effectively incorporates knowledge captured by specialist models.
Read in full here:
This thread was posted by one of our members via one of our news source trackers.