Course platform scaling
Course platform scaling and launch checklist
A course platform stays fast by paginating every list, indexing the main questions, and deferring heavy work until a measured trigger fires. At launch, apply 8 rules, watch 10 signals and hold the performance targets below.
Last checked
Rules to apply at launch
- Idempotency: Progress is one upserted row per (user_id, lesson_id), written at most every 15 s per lesson and on completion; completion is monotonic and idempotent. Course completion percent is a cached column on enrollments, so a dashboard with 50 courses is one indexed read.
- Pagination: Catalog, roster, discussion, gradebook and grading queue use keyset pagination on (sort_key, id) with page size <= 50; no OFFSET and no COUNT(*) on hot paths (counts are cached columns).
- Cdn: Lesson files, images and progressive video are served from the object bucket through the edge cache with signed short-lived URLs issued after canAccessLesson; asset names include content hashes so they are immutable and cacheable.
- Fanout: Synchronous exams create a thundering herd at start and end: attempt start and submit are single indexed writes, answers are autosaved client-side in batches, grading is O(questions) in memory, and nothing recomputes the whole gradebook per submit (gradebook is a projection updated per attempt).
- Rate limits: Per-user and per-IP rate limits on login, checkout creation, coupon redemption, quiz start/submit, discussion posts and certificate verification; coupon attempts capped per user per hour to stop code guessing.
- Search: Course search uses Postgres tsvector + GIN with a trigram fallback on title; a dedicated search engine is deferred until the catalog size trigger.
- Storage: Per-instructor storage quota and per-file size caps on uploads; per-plan limits on video minutes and egress; abandoned multipart uploads are aborted after 24 h.
- Data growth: attempts, submissions, discussion_posts and (phase 2) xapi_statements are the large tables: attempts and posts are indexed by course and date and archived by retention policy; xapi_statements are partitioned by month.
Performance targets
| What to measure | Target |
|---|---|
| Course landing page | LCP < 2.5 s; TTFB < 200 ms from edge cache |
| Lesson page access check | p95 < 100 ms, <= 3 queries (enrollment, cohort date, lesson) |
| Progress write | p99 < 80 ms, 1 upsert statement |
| Quiz submit and grade (50 questions) | p95 < 300 ms, graded in a single transaction |
| Learner dashboard | p95 < 150 ms for 50 enrollments, one indexed read |
| Catalog search | p95 < 200 ms at 100k courses via GIN |
| Certificate issued after completion | p95 < 60 s from the completing event to a downloadable PDF |
What to watch
- Metric checkout conversion funnel: landing view -> checkout started -> paid -> first lesson started; also webhook lag.
- Metric lesson access denied rate: 403s by reason (not enrolled, not yet released, refunded); a spike after a deploy means an access regression.
- Metric quiz submit latency and errors: histogram by assessment size; error rate during deadline bursts.
- Metric jobs queue depth and age: per job type; oldest certificate/email/scan job age drives alerts.
- Metric payout pending minor: payable earnings awaiting payout by instructor and platform total.
- Log structured learning and payment logs: request id, course id, user id hash, order id; no answers, submission contents, card or token data.
- Trace checkout -> webhook -> enrollment trace: one trace id from checkout creation to the enrollment row.
- Slo learning availability: 99.9% of lesson page and progress requests succeed; error budget reviewed monthly.
Recurring work
- issue_certificate: runs queue (jobs) when completion rules are met. Guard: unique (enrollment_id), re-checks completion and refund status, PDF size cap, attempt cap 3.
- run_instructor_payouts: runs cron weekly. Guard: only earnings past payable_at and not reversed, idempotent per (instructor, period), per-payout and platform daily ceiling, skips instructors without verified Connect onboarding.
- release_drip_notifications: runs cron hourly. Guard: bounded batch of enrollments whose next release time passed, one email per release, per-user daily email cap.
- rollup_course_stats: runs cron 5m. Guard: updates enrolled_count and rating aggregates in bounded batches; idempotent recompute from source rows for changed courses only.
- sweep_abandoned_orders: runs cron hourly. Guard: expires pending orders older than 24 h and releases cohort seats; never touches paid orders.
- scan_submission_files: runs queue after upload completes. Guard: virus scan and type verification in a sandboxed task; infected files are quarantined and the submission flagged.
Add when the numbers say so
- Read replicas: Route catalog browse, search, gradebook and analytics reads to a read replica once primary CPU > 60% sustained or read p95 > 200 ms; enrollment, checkout and attempt writes stay on the primary.
- Cdn: Adopt the video-streaming playbook (HLS ladder, immutable segments, manifest-time entitlement) for hosted video once video minutes or learner bandwidth make embeds or progressive MP4 too costly.
- Realtime: Live classes, live polling and presence run on a separate realtime tier in phase 3; phase 1 uses polling or SSE backed by Postgres only for notifications.
Deferred until it is needed
| Deferred | Adopt it when |
|---|---|
| Hosted adaptive video (HLS transcode pipeline) | hosted video > 500 hours or egress cost > 20% of revenue. |
| SCORM 1.2/2004, xAPI and LTI 1.3 | first institutional contract requiring packaged content or LMS integration. |
| KYC-backed guardian consent and school SSO (SAML/OIDC, roster sync) | under-13 learners or first school contract. |
| Live classes and proctoring provider | live sessions are a launch requirement. |
| Read replicas for catalog, gradebook and analytics | primary CPU > 60% sustained or read p95 > 200 ms. |
| Dedicated search engine (OpenSearch/Typesense) | catalog > 1M courses or search p95 > 300 ms. |
On Forge64
Forge64 matches your description to one of 12 reference architectures, writes the plan and data model into the project, generates the app, and runs a quality gate over the code before you see it. Each app gets its own Postgres database and sign-in with no per-user fee. Hosting on Forge Cloud and Forge Payments are rolling out.
Every metered resource on Forge64 has a hard cap, so a traffic spike stops at your limit instead of becoming a surprise bill. This is a reference design for the category, not a promise that every item is generated today. Built-ins such as email, background jobs, file uploads and search are coming soon, so anything here that needs them is something you add yourself for now.
FAQ
- When should I worry about scale for a course platform?
- Not before a measured trigger fires. For example, for "Hosted adaptive video (HLS transcode pipeline)": hosted video > 500 hours or egress cost > 20% of revenue.
- What performance targets should a course platform aim for?
- Course landing page: LCP < 2.5 s; TTFB < 200 ms from edge cache. Lesson page access check: p95 < 100 ms, <= 3 queries (enrollment, cohort date, lesson). Progress write: p99 < 80 ms, 1 upsert statement.
- Does Forge64 cap my usage costs?
- Yes. Every metered resource has a hard cap, and paid plans add an overage spend cap that you set.
Build your app with Forge64
One prompt to a running app with auth, database, hosting and payments. Start free, no credit card required.
Start building for free