I was able to run the 32B model on a 4090. It was pretty responsive. I would love to get my hands on a bunch of Mac Mini M4 Pro to compare the 4090 with a cluster of these.
1 Like
I was able to run the 32B model on a 4090. It was pretty responsive. I would love to get my hands on a bunch of Mac Mini M4 Pro to compare the 4090 with a cluster of these.