Loading...
From GitHub URLto Live URL
JustShip is a full-stack deployment orchestration platform — a mini Vercel — built from scratch using Docker, Redis, BullMQ, AWS S3, CloudFront, and WebSockets. Here's exactly how it works.
High-Level Architecture
User (Browser)→submits GitHub URL
Backend API→Node.js on EC2
Redis Queue→BullMQ job queue
Docker Worker→isolated build container
Build Output→/output/projectName
AWS S3→versioned static storage
CloudFront CDN→global delivery
Your Browser→project-name.just-ship.app
Step by Step
The Deployment Pipeline
What happens from the moment you hit Deploy to when your site goes live
01
GitHub Repo URL
You submit a GitHub repository URL through the frontend. The API validates it and creates a deployment record in MongoDB with status 'queued'.
02
BullMQ Job Queue
The API pushes a job into a BullMQ queue backed by Redis. This decouples the API from the build process — the API responds instantly while the job waits for a worker.
03
Docker Build Worker
A worker picks up the job and spins up an isolated Docker container. It clones the repo, runs npm install and npm run build, and writes output to /output/projectName. Container is resource-limited to 200MB RAM and 0.3 CPUs to prevent system crashes.
04
Upload to AWS S3
After a successful build, all output files are uploaded to S3 under a versioned path: project/v{n}/. Each deployment gets its own version — nothing is ever overwritten.
05
Served via CloudFront CDN
CloudFront sits in front of S3 and serves files globally. HTML files get no-cache headers so users always get the latest version. Hashed static assets are cached for 1 year — fast and cost-efficient with zero CDN invalidations needed.
06
Subdomain Routing
Every project gets its own subdomain: project-name.just-ship.app. A CloudFront Function parses the subdomain, maps it to the correct S3 path, and serves the right files automatically.
Under the Hood
Supporting Systems
The layers that make JustShip reliable, real-time, and production-grade
Webhook Event Routing
Each project can enable or disable auto deploy. GitHub webhook events trigger deployment jobs and push real-time in-app notifications to the right user session.
Real-Time Build Logs
The Docker worker emits logs to Redis Pub/Sub as it builds. A WebSocket server subscribes and streams them to your browser in real time — room-isolated per deployment so concurrent builds never mix logs. All logs are also persisted in MongoDB so you can review them after a refresh.
Version Switching
Every deployment is versioned and can be switched from the dashboard. Current live version is tracked separately from newly built versions.
Global CDN Delivery
Deployments are published to S3 and served through CloudFront. This provides edge caching, lower latency, and predictable static-hosting behavior worldwide.
Environment Variables
Project-level env variables are stored in MongoDB and injected into the Docker build at runtime. Transfers between frontend and backend are encrypted. Only authenticated users can deploy.
Deployment Reliability
Multiple deploy triggers prevented via localStorage lock. Failed deployment status correctly resets on refresh. Missing logs handled gracefully. JobId tracking fixed. Single active deployment enforced. Proper error messages shown — no generic failures.
Technology Stack
The Full Stack
Every tool used to build JustShip end to end
Node.js
Express
MongoDB
Redis (Pub/Sub)
BullMQ
Docker
AWS S3
AWS CloudFront
AWS EC2
NGINX
WebSockets
Next.js
TypeScript
Tailwind CSS
Framer Motion
The Builder
About Me
Hi, I’m Shyam. I’m a student who enjoys building backend systems and understanding how things work under the hood.
JustShip started as a small attempt to explore how modern deployment platforms work. While building it, I worked through queue systems, containerized builds, logging pipelines, and version handling.
Most of what I’ve learned came from debugging real issues and iterating on constraints — especially running everything on a small instance.
Source Code
The full source code for JustShip is available on GitHub. Feel free to explore the architecture, read the code, or raise issues.
github.com/shyam-3045/justship-api