Content site scaling
Content site scaling and launch checklist
A content site stays fast by paginating every list, indexing the main questions, and deferring heavy work until a measured trigger fires. At launch, apply 8 rules, watch 10 signals and hold the performance targets below.
Last checked
Rules to apply at launch
- Caching: Gated content is never in cached HTML, RSS, sitemap or search snippets: public HTML contains only the teaser, the full body is returned by an authed endpoint with Cache-Control: private, no-store after canRead, so one cached page cannot leak paid content.
- Pagination: Home, category, tag, author, comments, studio lists and search use keyset pagination on (published_at DESC, id) or (created_at, id), page size capped at 50; archives never use OFFSET.
- Queues: Newsletter delivery is a Postgres-queued, batched (100), idempotent job set with unique (issue_id, subscriber_id), a daily and monthly send cap checked before every batch, backoff on provider errors and an automatic pause on high complaint or bounce rates; an issue is never sent inline from a request.
- Rate limits: Per-IP and per-user limits on comments, subscribe, search, beacon, media presign and save; stricter limits for anonymous actors; over-limit gets 429 with Retry-After and is never queued.
- Search: Weighted tsvector + GIN and trigram titles, candidate cap 50, headline generation only for the top 10; identical queries are edge cacheable for 60 s; a dedicated search engine is deferred until the trigger.
- Pagination: Sitemaps are an index of child files with <= 5,000 URLs each generated from an indexed (status, published_at) scan and stored in the bucket, so a million-post site never builds one giant response.
- Data growth: page_views and newsletter_sends are partitioned and dropped on a retention schedule after rollup, autosave revisions are pruned, spam comments expire, and analytics read only rollup tables.
- Storage: Per-site media quota, max bytes and pixels per image, lifecycle expiry for pending uploads, source images deleted after variants verify when the plan says so, and unused assets listed for cleanup.
Performance targets
| What to measure | Target |
|---|---|
| Public post page (edge cache hit) | TTFB < 100 ms; LCP < 2.0 s on mobile; CLS < 0.05; INP < 200 ms |
| Public post page (cache miss, ISR regeneration) | server render p95 < 400 ms, <= 5 queries |
| Publish to live everywhere | p95 < 15 s from click to purged and revalidated |
| Editor save | p95 < 300 ms; autosave never blocks typing |
| Image delivery | edge hit ratio > 95%; hero image < 120 KB AVIF at 1280 w |
| Site search | p95 < 250 ms, <= 50 candidates |
| Newsletter throughput | >= 20 emails/s sustained within provider limits; 100k subscribers delivered in < 2 h, bounded by the daily cap |
| JavaScript shipped on reader pages | < 90 KB gzip; no client JS required to read |
What to watch
- Metric edge cache hit ratio: HTML and image hit ratio at the edge; a drop signals a cache-key, cookie or revalidation regression.
- Metric core web vitals: real-user LCP, CLS and INP per template, reported from the beacon.
- Metric publish to live latency s: time from publish to the first fresh edge response.
- Metric newsletter delivery: queued, sent, bounced and complaint counts per issue; daily send count versus cap.
- Metric comment spam rate: share of comments scored or flagged as spam; moderation queue age.
- Log structured publish, revalidate and send logs: request id, post id, actor id hash, issue id; no emails, tokens or gated body text.
- Trace publish -> revalidate -> purge trace: one trace id from the publish click through edge purge confirmation.
- Slo reader availability: 99.95% of public page requests succeed (served from edge or origin) under 1 s; error budget reviewed monthly.
Recurring work
- publish_scheduled: runs cron 1m. Guard: claims posts with status=scheduled and publish_at <= now using SKIP LOCKED, bounded to 100 per run, idempotent status flip, then enqueues revalidation.
- revalidate_paths: runs queue (publish, edit, slug or collection change). Guard: deduplicates paths per run, caps purges per minute, always includes home, collection, tag, feed and sitemap.
- process_image: runs queue (media_assets pending). Guard: resource-limited sandbox, magic-byte check, MAX_IMAGE_PIXELS and MAX_UPLOAD_MB caps, EXIF stripped, attempt cap 3, minutes metered into usage_events.
- schedule_newsletter: runs cron 1m. Guard: materialises newsletter_sends in chunks of 5,000, refuses when recipient_count exceeds the daily or monthly send cap, marks the issue paused with a reason.
- send_issue_batch: runs queue (newsletter_sends queued). Guard: batch of 100, unique (issue, subscriber), skips suppressed addresses, NEWSLETTER_MAX_SENDS_PER_DAY and monthly cap checked before each batch, auto-pause on complaint rate > 0.1% or bounce rate > 5%, attempt cap 3 with backoff.
- rollup_page_views: runs cron 10m. Guard: aggregates only partitions touched since last run, upsert by (post_id, day), bounded batch.
Add when the numbers say so
- Read replicas: Route studio lists, search and analytics reads to a read replica when primary CPU or read latency crosses the trigger; public reads already bypass the database through the edge cache.
- Cdn: Add regional origin shielding and multi-region object replication for global audiences when origin egress or cache-miss latency crosses the trigger.
- Realtime: Collaborative editing and live preview over websockets arrive in a later phase; phase 1 uses preview links and optimistic rev conflict handling.
Deferred until it is needed
| Deferred | Adopt it when |
|---|---|
| Dedicated search engine (OpenSearch/Typesense) | > 2M indexed posts or search p95 > 250 ms. |
| Read replicas for studio, search and analytics | primary CPU > 60% sustained or studio read p95 > 300 ms. |
| Translations (hreflang) and versioned docs | first multilingual customer or a docs site with a second major version. |
| Collaborative real-time editing | > 3 concurrent editors per post is a common workflow. |
| Podcast and audio/video hosting with per-episode analytics | audio is a launch requirement; video uses the video-streaming playbook. |
| Ad and sponsorship serving | ads are the primary revenue model. |
On Forge64
Forge64 matches your description to one of 12 reference architectures, writes the plan and data model into the project, generates the app, and runs a quality gate over the code before you see it. Each app gets its own Postgres database and sign-in with no per-user fee. Hosting on Forge Cloud and Forge Payments are rolling out.
Every metered resource on Forge64 has a hard cap, so a traffic spike stops at your limit instead of becoming a surprise bill. This is a reference design for the category, not a promise that every item is generated today. Built-ins such as email, background jobs, file uploads and search are coming soon, so anything here that needs them is something you add yourself for now.
FAQ
- When should I worry about scale for a content site?
- Not before a measured trigger fires. For example, for "Dedicated search engine (OpenSearch/Typesense)": > 2M indexed posts or search p95 > 250 ms.
- What performance targets should a content site aim for?
- Public post page (edge cache hit): TTFB < 100 ms; LCP < 2.0 s on mobile; CLS < 0.05; INP < 200 ms. Public post page (cache miss, ISR regeneration): server render p95 < 400 ms, <= 5 queries. Publish to live everywhere: p95 < 15 s from click to purged and revalidated.
- Does Forge64 cap my usage costs?
- Yes. Every metered resource has a hard cap, and paid plans add an overage spend cap that you set.
Build your app with Forge64
One prompt to a running app with auth, database, hosting and payments. Start free, no credit card required.
Start building for free