Design: uploading images and 2 GB lecture videos Intermediate

2 GB lecture videos currently pass straight through the Node.js API, land on the pod’s disk and get transcoded inside the request. The empty board is the current system; drag components into the four layers to replace each path: upload, storage, processing, playback.

• Goal: meet every requirement within the complexity budget.
• Try also: find one design that uses signed cookies and one that uses signed URLs. Then try raising the ALB timeout instead of removing the proxy path, or replacing the worker with Lambda, and see which metric turns red.

Design

Design uploads for images and 2 GB lesson videos

Teachers upload lesson videos (up to 2 GB) and images. Today: the Node.js API receives the whole file, writes it to the pod's disk, runs ffmpeg inside the request, and then the same API pods stream the video to students. The result: the ALB returns 504, pods run out of RAM and disk, files vanish after deploys, and distant students wait forever for the picture.

The API (3 pods, Singapore) and PostgreSQL already exist. The empty board is the current system; you add components to replace each path: upload, storage, processing, playback.

Assumptions for the game: peak of 40 concurrent uploads, teacher uplink ~20 Mbps so 2 GB takes ~14 minutes; videos are 60 minutes long; 3,000 students watch at the same time in the evening, some in Australia and the US. Lost/stuck counts are per 1,000 videos over a month with ~20 deploys. Illustrative numbers, not benchmarks.

Requirements

• 2 GB upload success ≥ 99%
• The API holds no long connections for uploads or playback
• 0 lost or stuck videos / 1,000
• From upload finished to watchable ≤ 20 minutes
• Video startup p95 for distant students ≤ 2 s
• 0 ways for outsiders to watch paid lessons
• Complexity ≤ 12 units

Press ＋ Add on each layer to choose a component. On a computer you can drag a component onto a layer. Metrics update right away.

👤 Parallel multipart upload🌐 CDN + signed cookie🌐 CDN + signed URL🌐 Public CDN, hard-to-guess URLs🛡️ Raise ALB timeout to 30 minutes⚙️ Issue presigned URLs⚙️ POST /media/:id/complete⚙️ 2 GB body limit, proxy through API⚙️ Base64 file in JSON⚙️ ffmpeg in the request handler⚙️ API pipes video from bucket🗄️ Private bucket (S3/GCS)🗄️ Lifecycle: abort incomplete multipart📨 S3 event notification🗄️ Media table with status🗄️ Public-read bucket🗄️ Store files in the pod emptyDir📨 SQS queue + DLQ🔧 Transcode worker (HLS)🔧 Virus scan + ffprobe🔧 Lambda transcode from S3 event

Tap a component to see what it does.

Users👤 Client

Browser & CDNClient-side upload logic, CDN in front of storage＋ Add

Lesson API⚙️ Lesson API (3 pods)🔒＋ Add

Storage🗄️ PostgreSQL🔒＋ Add

Background processingQueues and workers running outside the request＋ Add

Long connections on API3,040

2 GB upload success40 %

Lost/stuck videos / 1,00090

Upload done → watchable35 min

Video startup p955 s

Paths exposing private content0

Complexity0 units

• Uploads still go through Node.js as they do today: 40 concurrent uploads hold 40 long connections, and requests > 5 minutes are cut off by Node (requestTimeout defaults to 300 s). Only ~40% of 2 GB uploads succeed.
• Files live on the pod's disk (emptyDir or node disk): when a pod is deleted or moved during a deploy, the files are gone; each file exists on only 1 of 3 pods, so the other 2 return 404. Assume ~30/1,000 videos lost per month.
• ffmpeg in the request handler: after the upload, the request sits silent for ~30 minutes waiting for the transcode; the ALB sees no bytes for more than 60 s (the default idle timeout) and returns 504. ffmpeg hogs the pod CPU, and other requests slow down too.
• Videos play from the API pods: 3,000 students at peak hold 3,000 streams across 3 pods in Singapore. Students in Australia and the US wait ~5 s before the picture shows.

↺ Start over

Why pushing large files through Node.js eats RAM and holds connections, and how far streams help: Streams and backpressure.