Husky: a model-specific inference engine up to 4.5× faster than Apple's MLX

macOS Developer Forum macOS In The News
, , , ,
1

On a MacBook, Woof runs up to 730 tokens a second with Husky Flash on, and answers 5× sooner with the same weights. Live in Underdog today.

Read in full here: