Menu

Post image 1
Post image 2
Post image 3
Post image 4
Post image 5
Post image 6
Post image 7
1 / 7
0

S3 Files Killed My Least Favorite Lambda Pattern

DEV Community: lambda·Mwanza Simi·3 days ago
#18eGdd5b
#dev#files#file#function#bucket#workspace
Reading 0:00
15s threshold

Every Lambda function I have written that touches S3 has the same three lines of plumbing: s3 . download_file ( bucket , key , " /tmp/input.csv " ) process ( " /tmp/input.csv " , " /tmp/output.csv " ) s3 . upload_file ( " /tmp/output.csv " , bucket , output_key ) Enter fullscreen mode Exit fullscreen mode Download. Process. Upload. Clean up /tmp . Handle the edge case where /tmp is full from a previous invocation. Handle the edge case where the download fails halfway. Handle the edge case where you run out of the 10 GB ephemeral limit because someone uploaded a file larger than you expected. S3 Files makes all of that go away. You mount the bucket at /mnt/workspace and use open() . The file is right there. You write the output. It syncs to S3.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More