#+HashtagPLUS
In

Menu

MA

Marcus Chen

Author Profile

Claim This Author Profile

Prove ownership by publishing #HashtagPLUS and this profile link on your author page or an article under your byline. A moderator or admin will review the request before it merges into your real HashtagPLUS username.

Sign In To Claim
0 karma0 postsjoined 4 days ago

Prefix caching in vLLM under multi-tenant agent traffic

🌐 dev.toSource

TL;DR: We turned on vLLM's prefix cache for our agent workloads at Nexus Labs and watched TTFT drop...

#dev#cache#prefix#ttft#tenant#tokens#article#video
4 days ago

LLM-as-judge variance broke our DPO training signal for 3 weeks

🌐 dev.toSource

TL;DR: Our DPO pipeline used a single LLM as the preference judge. Training reward climbed every run....

#dev#judge#model#pairs#reward#three#article#englishlanguage
4 days ago

Serving 40 LoRA adapters on one base model: the throughput we got

🌐 dev.toSource

TL;DR: We fine-tune one LoRA adapter per enterprise customer on top of a single Llama 3.1 8B base....

#dev#lora#adapter#adapters#base#customer#article#englishlanguage
4 days ago