Menu

Post image 1
Post image 2
Post image 3
1 / 3
0

Background AI: Using Solid Queue for Slow OpenAI API Calls

DEV Community·Zil Norvilis·20 days ago
#iDdvAmGA
Reading 0:00
15s threshold

Very often I see developers integrating AI into their Rails apps for the first time, and they make a critical mistake that completely destroys their server performance. They treat the OpenAI (or Anthropic) API like a regular database query. They put the API call directly inside their controller. # The "Server Killer" Approach def create @document = Document . find ( params [ :id ]) # This might take 15 seconds! response = OpenAiClient . generate_summary ( @document . text ) @document . update ( summary: response ) redirect_to @document end Enter fullscreen mode Exit fullscreen mode When you do this, the Puma web thread handling that user's request freezes . It sits there doing absolutely nothing for 15 seconds while it waits for the AI to respond. If you have 5 users asking for summaries at the same time, your entire server will lock up. No one else will be able to load your website. The browser might even time out and show an error page. AI calls are slow. You must put them in the background.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More