Vibe Code Bench 1-100

Background

Most coding benchmarks stop at the first working version. Vibe Code Bench 1-100 measures what comes after: whether a model can evolve a working web application through a long sequence of dependent product requests without breaking what already works.

It picks up where Vibe Code Bench (VCB) leaves off. VCB measures zero-to-one generation, starting from scratch and producing a working web application. VCB 1-100 begins with a complete, fully passing web application from Vibe Code Bench v1.1 and asks the model to keep going.

Each task hands the model an ordered sequence of realistic requests without specific technical details, the kind a PM sends an engineer or a vibe coder types at an agent. Later requests depend on earlier features, data models, permissions and user journeys. The model keeps both its conversation and its code across the whole task. Every iteration has to add the new behavior while preserving all the behaviors before it, and the first incomplete iteration ends the attempt.

This matters because most real software work is not greenfield. It is shipping the next feature, migrating a schema without losing data, applying a policy change consistently across every surface of an app. It is the skill no one-shot benchmark touches, and the one that decides whether coding agents can be trusted with production codebases.

Overview

Long-horizon and realistic product requests

VCB v1.1 measures building an application from a specification. VCB 1–100 starts with a working application and asks the agent to extend it through up to ten dependent requests.

The requests describe product behavior that needs to be changed in plain English, similar to how developers prompt while vibe coding, without any technical jargon. The agent must locate the relevant code, understand existing systems, and choose an implementation of the feature on their own. Later turns retain the code, data, and conversation from earlier work. A short request can require substantial engineering, as compatibility with previously added features needs to be preserved.

Regression tests ensure that the app stays usable under development

Each iteration includes tests for the new request and selected regression tests for earlier behavior that must remain active. Both sets of tests run against the same submitted candidate app. If a model successfully adds a new feature, but breaks the existing app features in the process, we still consider that a failure.

About one third of all workflows in the dataset are regression checks: 3,747 out of 11,299 (33.2%), with the remaining 7,552 (66.8%) covering new feature checks. These numbers span the same 939 requests across 100 application arcs, including the test set and the validation set.

The regressions are a selected subset. At iteration 2 they cover 43.1% of the new request workflows from iteration 1. Later iterations still spend about 30% of each suite on regressions, but they do not re-check every earlier flow.

Methodology

A sequence of changes to one application

Each task starts with a working Vibe Code Bench v1.1 application and its saved app state. The model receives up to ten product requests, one at a time. It edits the same codebase throughout and retains its conversation and tool history. Later requests build on earlier features and decisions; they are not independent coding tasks.

After each change, a browser agent checks the new request and selected regression workflows against the submitted application. Every step must pass for a workflow to pass. The reported score counts consecutive completed iterations as long as they pass with 100% workflow success, stopping at the first incomplete or incorrect iteration. Breaking previous features (from previous iterations or the base app) also counts as failing.

All models use the same modified OpenHands V1 environment, with terminal access and preconfigured Supabase, Stripe test services, and MailHog. Each task has one ten-hour generation budget shared across its iterations. Evaluation time is excluded.

Task construction

Each task consists of a working starting application, an ordered sequence of product requests, and browser-based acceptance tests for each iteration.

Requests build on the product state left by earlier changes. They specify the desired behavior and preservation constraints without prescribing an implementation. The sequence covers five change types and contains up to ten iterations.

Each iteration includes new feature tests and selected regression tests. The tests assess observable behavior rather than require a particular internal code structure. The dataset contains 50 test set scenarios and 50 separate validation scenarios. All model scores reported on this page are only over the test subset.

Five kinds of software change

In software, the tasks cover many different types of changes that will be needed in producing and iterating on a real application.

Our feature requests are categorized into five types of changes, listed below with examples.

Cross-cutting changes form the largest group. Success requires applying a rule consistently across the product, not only adding a screen or endpoint. Migrations and reverts test a different skill: changing or removing functionality while retaining the data and behavior that must survive.

Quality Assurance

Tasks undergo expert review for clarity, realism and consistency. The experts also review the iteration checks to ensure that expected behavior in the checks follow from the request and that all reasonable implementations can satisfy the tests.

Each workflow must have reproducible setup, actions a browser can perform, and a clear result.

The review checks four things:

• Prompt/test agreement: Every tested expectation is supported by the request, and each core requirement has a corresponding check.
• Executable workflows: The steps can be performed in a browser, with enough setup information and a clear yes-or-no result.
• Realistic scope: The request gives enough information for a useful implementation without prescribing an unnecessary technology stack.
• Preservation and edge cases: The requested change does not contradict earlier live behavior, and important unhappy paths are covered.

Task review and execution checks serve different purposes: one checks what should be tested; the other checks that the application and test can run. Missing or incomplete observations are not treated as proof of an implementation defect.

Example task: Zeeter

Zeeter is a small social application with accounts, posts, profiles, likes, and follows. This standalone example is a separate task, not part of the test or validation sets.

Example request — iteration 1

Add a private Saved area where signed-in members can save and unsave posts from the feed, profiles, and post detail, then find their saved posts without changing likes or follows. Each saved post can have a private note of up to 500 characters; repeated saves must produce one entry, and saved posts must show the current source text after edits. Guests and other members must not see a member’s Saved area or notes.

Example acceptance workflow — save from the feed

1. Open Zeeter. Sign out if needed. Create username 'zt1w1owner', email 'zt1w1owner@example.com', password 'ZeeterPass123!'. Complete or skip profile setup to reach the home feed.

2. For this workflow only, the separate reader account is username 'zt1w1reader', email 'zt1w1reader@example.com', password 'ZeeterPass123!'. Create it only when instructed; later sign-ins use these same credentials. The original author is 'zt1w1owner' with password 'ZeeterPass123!'.

3. Publish a post with text 'zt1w1alpha source post'. Save it from the home feed.

4. Open Saved and confirm exactly one entry for 'zt1w1alpha source post'. Reload and confirm it remains.

In the example below, this workflow passed all four steps. Other workflows check note privacy, repeated saves, and existing interactions. Later regression checks include a previously shared URL: removing its button is not enough if guests can still open the URL.

Note that in real tasks from the test set the workflows are longer and test more complicated aspects of the changes, such as edge case interactions.

Example generated output

The image below shows the Zeeter starting app during a browser replay.

The evaluation GIF below shows this save-from-feed workflow after iteration 1.

Evaluation

We use Browser Use to evaluate the submitted application through its user interface. The evaluator agent follows each workflow and checks the expected behavior. Each iteration includes checks for the new request and selected regression checks for earlier functionality.

A workflow passes only when every step succeeds. The reported score credits an iteration only when all its workflows pass. We stop the task after the first incomplete iteration - if an application is broken we do not attempt to add more features on top.

A scenario’s score is the number of consecutive iterations passed, divided by the number planned for it. The benchmark score is the mean across scenarios. All models run on the same OpenHands harness, against the same scenarios and the same checks.

Results

How many iterations each model completes

Within a single model’s runs, the ability to reliably modify the existing application varies sharply. Even the leading models have many tasks where they fail at the first request in the existing app. Sustaining several changes is doable for many models, but none do so consistently across all tasks.

Performance by change type

Across the reange of models, reverts have the highest pass rates and schema migrations the lowest. Removing functionality while preserving required behavior is easier for the agents to perform correctly without breaking other features than changing the data model beneath a working application.

Note: The models reach different iteration requests depending on whether they pass previous tasks, so the comparisons across models is not necessarily exact.

Failure to implement vs. Breaking what already works

Implementing the new request is only one part of maintenance. A change often damages behavior that worked before, even when the new feature is implemented correctly .

Most recorded workflow failures involve new request checks alone. Regression failures also contribute substantially, either on their own or alongside broken new features. Preserving existing behavior remains a separate challenge from implementing new features.

How models use their tools

Opus and Fable use the browser more frequently than Astra. Astra relies more on explicit task tracking and also runs browser automation through the terminal. This shows that similar overall scores can emerge from different working styles.

Where the model inspects the running app varies significantly. Opus and Fable put about one third of their calls in the browser. Astra puts about one ninth there and uses the task tracker more than either of them: 11 tracker calls per iteration, versus 3.7 for Opus. Grok spends most of its calls in the file editor and almost none in the browser. Gemini Pro and Inkling also rarely open the native browser, and all three sit in the bottom third of the table. Astra is the exception among the high scorers: it checks its work with HTTP requests from the shell rather than by driving the browser.

Refusals and fallbacks

Only two models had refusals (task was rejected completely) or fallbacks (the task was routed to another model for completion). Qwen 3.8 Max refuses and the task stops there as there is no fallback enabled. Claude Fable 5.1 refuses in 14 tasks and all 14 continue, with Claude Opus 5 taking over the refused turn. Occasionally Fable 5.1 also falls back to Opus 4.8, but this covers only a couple of turns across the whole benchmark. On all tasks, the refusals eventually stop and Fable 5.1 takes back over before iteration submission.

Qualitative analysis

Failure modes

Failures often cross feature boundaries. A wrong record association or inconsistent ledger can affect both a new workflow and an existing one. Other failures stay local: a missing control may block the new request while earlier behavior remains intact. Failed check groups show where behavior breaks, but do not identify the cause on their own.

Among failures on new checks alone, reviewed examples include missing controls, privacy leaks, duplicate writes, and state assumptions. The mixed example shows accounting and record association failures affecting both new and regression checks.

Astra blocks itself on resolvable clarification

One distinct failure mode that we see across multiple tasks is that Astra asks for clarification on certain points and blocks without receiving results. This causes a failure, as the system prompt specifies that the model cannot ask questions of the user, and there is no interaction with the user included in the benchmark. The specific clarification points Astra asks about are points that the model is either expected to be able to understand the correct behavior on its own, or are points where it is ensured that any reasonable implementation will pass the tests (no tests are added for interactions which are unspecified). For example, in a request to add a persistent edit history summary to a ledger, the model requested clarification on whether the delete all data function should also delete edit history. This exact interaction is not tested in the iteration checks, and either option would be acceptable. The model does not continue without receiving an answer and did not complete implementation, causing it to fail the task despite having a plan for the iteration that may have had the ability to succeed.

Multi-iteration trajectories across models

This shows an example of how each model works through one fixed reference task, iteration by iteration. Most interleave editing with browser checks, though the timing differs: Opus 5 and Fable 5.1 spread their browser use evenly across an iteration, while Gemini 3.8 Flash and Kimi K3 push most of it late. The mix also shifts from one request to the next, as GPT-5.6 Terra goes from half its calls in the browser at the first iteration to none by the sixth, while MiniMax M3 works without the browser until the last.

Note that some models stop early due to failing a check or hitting the generation time limit.

Citation

Please cite this work as:

@article{valsai2026vibecodebench1100,
title        = {Vibe Code Bench 1-100},
author       = {Hung Tran and Dilhan Salgado and Langston Nashold and Orestes Kassian and Rayan Krishnan},
year         = {2026},
month        = sep,
howpublished = {Vals AI},
url          = {https://vals.ai/benchmarks/vcb-1-100},
}