Forge64 logo
Trust Center

Quality

How Forge64 verifies generated apps

Last reviewed

Forge64 does not trust the model's description of its own output. A gate reads the files that were produced and applies deterministic security, architecture and cost rules to them, a static audit checks the app (for example that user-data tables enable row-level security), and failed checks feed a bounded repair loop. The build, type, lint and test lanes are defined but depend on the app's own scripts, and the end-to-end, visual and accessibility lanes are not yet run in continuous integration. A benchmark of 30 prompts and 120 criteria is built to measure quality against other builders, but no scores are published because no live run has been completed.

The gate: it reads the files, not the model's claims

The gate's lane list covers typecheck, lint, unit, contract, integration, end-to-end, visual, accessibility, security, architecture, cost guard and capability scenarios, plus pre-build gates that stop a build if a promised capability has no implementation on disk. Not every lane can grade a generated app held in memory. The lanes that grade the artifact's own bytes today are the static security, architecture and cost-guard rule sets and the capability resolver trace. The npm-script lanes run against the generated app's own package scripts.

  • BuiltArtifact-scoped lanes (static security, architecture and cost-guard rules, capability resolver trace) grade the generated files themselves.
  • BuiltPre-build gates abort generation when a promised capability's implementation is missing.
  • Rolling outTypecheck, lint, unit, contract and integration lanes run through the app's own scripts. Whether every production generation executes all of them has not been verified end to end.
  • RoadmapEnd-to-end, visual and accessibility lanes exist and run when invoked, but continuous integration does not call them yet.
  • BuiltA baseline template and a manifest of required items that generation must keep, with a bounded automatic repair loop.

The audit loop

A catalog of deterministic static audit checks runs on the generated app, including that every user-data table enables row-level security. Failed checks produce specific repair instructions, and the repair loop is bounded, so it cannot run forever or spend without limit.

  • BuiltDeterministic static audit checks with fix instructions, including row-level-security checks on generated tables, available as the project security scan.
  • BuiltA bounded repair loop with per-stage token and cost budgets.
  • Rolling outAn optional model-based judge layer on top of the deterministic checks.

The benchmark method

The benchmark corpus has 30 realistic prompts across 10 categories, with 120 acceptance criteria: 90 automated (browser flows) and 30 manual rubric items for things a crawler cannot judge, such as server-side enforcement and arithmetic correctness. Each generated app is scored on gate verdicts, build and type checks, product checks (sign-up, sign-in, billing page, five screen widths, security headers) and performance and accessibility audits. A score that was not measured counts as missing and lowers coverage; it never raises the score. A regression gate fails if any category or app score drops.

To compare builders, the same prompt is entered verbatim into each product with no follow-ups, the timing is recorded and the result is imported through the same checks. Builders are ranked only on the components that were measured for every builder. Forge64's own gate is never applied to a competitor's code.

  • BuiltBenchmark corpus, runner, scorecard and regression gate, tested against recorded (mock) runs.
  • RoadmapA live benchmark run and a competitor comparison. Neither has been completed, so Forge64 publishes no score and makes no claim to outperform any builder.

What verification cannot promise

Automated checks catch build failures, missing tables and common security mistakes. They do not prove that an app does what you meant. Review the app, test the flows that matter to your business and read the security findings before you take payments.

Frequently asked questions

How good are the apps Forge64 generates?
Forge64 applies a gate and a static audit to what it generates, but it has not published benchmark scores because the live benchmark run has not been completed. Expect a working full-stack starting point that you should review, not a guarantee that it matches your intent.
How does Forge64 check generated code?
A gate reads the produced files and applies static security, architecture and cost-guard rules and a capability trace. A static audit checks the app, including that user-data tables enable row-level security. Type, lint and test lanes run through the app's own scripts, and end-to-end, visual and accessibility lanes are not yet run in continuous integration.
Is Forge64 better than Lovable, Bolt, Replit or Base44?
Forge64 makes no such claim. A benchmark method for comparing builders on the same prompts is built, but no live comparison has been run. The Trust Center compares hosting and data facts only.
Does Forge64 fix its own generation mistakes?
Yes, within limits. Failed audit and gate checks produce repair instructions and a bounded repair loop retries a limited number of times with a token and cost budget, then stops.