Menu

Post image 1
Post image 2
Post image 3
Post image 4
1 / 4
0

My RTX 5090 can't keep up with Apple Silicon on the biggest local LLMs, and I hate to admit it

XDA·Adam Conway·19 days ago
#PPHKZr92
#sensa#ai#community#model#apple#memory
Reading 0:00
15s threshold

I spent a long time building the gaming PC I wanted, iterating over the last decade and finally landing on a PC that the younger me could have only dreamed of. I've got an Nvidia RTX 5090 and an AMD Ryzen 7 9800X3D, and it handles every game that I throw at it without breaking a sweat. On top of that, I do a lot of local heavy computational workloads, like machine learning, data analysis, and development. However, as local LLMs have taken off, I've been playing around with them and seeing what they can do. I now run them every day, and while I had thought the RTX 5090 would be an incredible beast capable of running them at impossible speeds, I realized something very quickly: it's fast, but speed isn't all there is. Granted, Qwen 3.6 27B is a phenomenal model, and it fits nicely in the 32GB of VRAM that the RTX 5090 has. But there are other, more interesting models that I'd love to try out, but those are significantly larger than what I can fit in a mere 32GB pool.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More