Menu

Post image 1
Post image 2
1 / 2
0

I Tried to Compress an LLM by 545x. Here's What Happened

DEV Community·HasanH47·29 days ago
#i18IgBpq
Reading 0:00
15s threshold

A solo dev's journey questioning a 40-year-old assumption in deep learning The Question That Started It All I was frustrated. VS Code was getting heavier on my laptop. Cursor wanted $20/month. The best AI agents were owned by 5 mega-corporations. As a developer in Indonesia, I sometimes felt we were perpetual consumers, never creators. So I asked Claude: "Can AI be smaller?" That conversation led somewhere unexpected. We started questioning the most fundamental assumption in deep learning since 1986: Do weights have to be stored as matrices of numbers? Think about it. A human brain doesn't store information as numbers. A seed doesn't contain all the leaves of a tree inside it — a seed contains instructions to grow leaves. What if AI weights could be grown from a small seed when needed, instead of stored as massive matrices? A 30B model could fit on a smartphone. No cloud needed. No subscription. No billion-dollar hardware. I named the project WIJI — "seed" in Javanese.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More