Menu

Post image 1
Post image 2
1 / 2
0

Delivering E2EE media without blowing up Postgres

DEV Community·Martin Kambla·30 days ago
#PPCpJLLW
Reading 0:00
15s threshold

When I launched my messenger, the media upload path looked like this: client → encrypt → POST /media/upload → INSERT INTO media (ciphertext BYTEA) Enter fullscreen mode Exit fullscreen mode Functionality was there, 2MB, 25MB (sometimes times out) and at 100MB you get blowups. Here's a brief "lessons-learned" about the road of Postgres BYTEA at scal, and the architectuer I ended up with shipping 200MB encrypte video without the server ever seeing plaintext. Why BYTEA was good at first? I've always preferred Postgres to any other DB system. I'm sure others have their benefits in different scenrios but I usually when dealing with serious applications I use Postgres. And for messaging, the attraction was specific. If the ciphertext lives next to the row pointing at it, there's one write, one transaction, one thing to fail. No dangling references between the DB and a blob store if one side of the write fails. Simple. What breaks? Then I tried to ship a 50 MB video. WAL explosion.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More