SLICC · benchmark results · updated 12 October 2026
Browser Agent Benchmark: Claude vs. GPT
AI models are getting good at using a browser, but their scores and costs are far apart. On BU Bench V2.1, Browser Use's set of 200 long, multi-step web tasks, GPT-6.1 Sol at high effort scores 60.0 out of 100 for $2.22 per task. Claude Opus 5.5 at max effort scores 53.5 for $5.21. Across the 17 configurations that ran at least 188 of the 200 tasks, scores range from 28.1 to 60.2, and cost from $0.018 to $5.21 per task.
We add new models as they land. Since 29 September this page has added Claude Sonnet 5.5, GPT-6.1 Sol at four effort levels and, on 8 October, runs with a small decision model in the harness. On 9 October it added Claude Haiku 5.5 at three effort levels; at low effort it is the cheapest configuration on the page, 31.9 for $0.018 per task. On 12 October it added Grok 4.7 at low and high effort. The updates list what changed each time.
A small decision model brings the cost down. Some decisions never need to reach the large model: which button on the page to click, or which line of it holds the price. With SLICC's new intent skill, the large model states each browser step in one sentence and a small model, running locally inside the browser it drives, carries it out. GPT-6.1 Sol at low effort then costs $0.21 per task instead of $0.80, 74% less, and scores 40 instead of 45 on the same 196 tasks. The GPU's cost is not included. How it works and what it costs in score.
Every run on this page ran inside SLICC, an open-source agent that works in your own browser with the model you choose. Run it on your own tasks.
Ranking and score vs cost
Score is the share of each task's rubric that the judge marked as met, averaged over judged runs and scaled to 100. It is not the share of tasks solved. Cost is the average model spend per task at list prices. Runs is the number of judged runs behind a score. The line joins the configurations that no other configuration beats on both score and cost. It runs through Claude Haiku 5.5 at low, medium and max effort, GPT-6.1 Sol at low effort with and without intent, Claude Sonnet 5.5, and GPT-6.1 Sol at default, high and max effort.
Seventeen configurations ran 188 to 200 of the tasks. Five ran only a 20-task screening round and are drawn hatched and hollow: treat them as indicative. Each full-run row counts one run per task. The two rows with intent ran 3 to 8 days after the runs they are compared with, on a newer SLICC version, and their time per task is not comparable; see the intent section. Claude Haiku 5.5 ran on 8 and 9 October, 7 to 11 days after the other full runs without intent, on SLICC 6.242.1. Grok 4.7 ran on 9 October, 8 to 11 days after them, on SLICC 6.245.0.
Quick start
npx sliccy
Downloads SLICC, launches Chrome, and opens the workspace. Requires Node 22+.
Global install
npm install -g sliccy
Installs SLICC globally. Run slicc from any directory.
Desktop & Extension
- https://www.sliccy.ai/download/slicc.dmg
- https://chrome.google.com/webstore/detail/akjjllgokmbgpbdbmafpiefnhidlmbgf
macOS app: download, drag to Applications, double-click. Chrome extension: side panel agent with access to your logged-in sessions.
A small model for the small decisions
Many of a browser agent's steps are small: click the search box, type a name, read a total. Without intent, the large model reads a snapshot of the page and picks every control itself. The intent skill splits the work. The large model still decides what to do next and says it in one sentence, such as "choose the Medium pizza size". A small vision model, kev-4b-vision (5.4 GB), finds the control or the text that sentence means and acts on it, and the step returns a few lines instead of a page snapshot. When the small model is not sure which control is meant, it does nothing and returns the likely candidates, so the large model can be more specific. Asked to read something, it returns the closest matching text.
On the same tasks, with intent against without:
- GPT-6.1 Sol at low effort: 40 against 45 (-10.3%), 74% cheaper. n = 196.
- Claude Sonnet 5.5 at low effort: 30 against 35 (-13.5%), 41% cheaper. n = 185.
- GPT-6.1 Sol at low effort with intent against Claude Opus 5.5 at low effort: 40.5 against 37.0 (+9.5%), 50% cheaper. n = 190.
- GPT-6.1 Sol at low effort with intent against Claude Sonnet 5.5 at default effort: 41 against 51 (-19.6%), 83% cheaper. n = 183.
The cost saving holds up. The score loss depends on the model, and it is less certain, because the intent runs ran on SLICC 6.238 from 4 to 6 October, 3 to 8 days after the runs they are compared with, and scores drift between days. On 40 of the tasks, three configurations ran with and without intent on the same day, and no difference was clear: each 95% interval includes zero. Adjusted for the drift measured on those 40 tasks, Claude Sonnet 5.5 at low effort loses about 1 point with intent (95% interval -7 to +6 points), which is within noise. GPT-6.1 Sol at low effort loses about 11 points (-20 to -2), more than the 5 points of the raw comparison.
Two more limits. The cost per task covers the large model only, not the GPU that runs the small one. And runs with intent finish sooner, but by less than the table suggests: runs without intent end with a 2-minute settle wait that the intent runs skip. Without that wait, Claude Sonnet 5.5 at low effort is 18% faster with intent and GPT-6.1 Sol at low effort 33% faster.
The intent skill is open source, in ai-ecoverse/skills.
Claude Haiku 5.5 ran all 200 tasks on 8 and 9 October at three effort levels: 31.9 at $0.018 per task at low effort, 38.6 at $0.095 at medium and 48.4 at $0.935 at max. All three join the score-vs-cost frontier, and Claude Sonnet 5.5 at low effort, with and without intent, leaves it; Haiku with thinking off ran only on a 40-task screening set.
GPT-6 Luna now scores 28.1 at $0.059 per task, after 7 runs that used Claude Sonnet 4.6 sub-agents were re-run with GPT-6 Luna only. Claude Opus 5.5 at default effort now counts one run per task, like every other full run, and scores 50.6 at $1.556.
Two new full runs use SLICC's intent skill, in which a small model on a GPU carries out the browser steps the large model describes. GPT-6.1 Sol at low effort scores 40.5 at $0.210 per task with intent, against 45.1 at $0.799 without. Claude Sonnet 5.5 at low effort scores 29.6 at $0.101 per task, the cheapest configuration on the page. [Corrected 9 October: GPT-6 Luna, at its corrected $0.059 per task, was cheaper.]
Both join the score-vs-cost frontier, and Claude Opus 5.5 at low effort leaves it: on the same 190 tasks, GPT-6.1 Sol at low effort with intent scored 40.5 against 37.0 at half the cost. The intent runs are 3 to 8 days newer than the runs they are compared with, and their cost leaves out the GPU.
GPT-6.1 Sol now has full runs at low, high and max effort. At high effort it scores 60.0 at $2.224 per task. On the 168 tasks where both have a judged run, it scored 60 against 55 for Claude Opus 5.5 at max effort, 61% cheaper and 45% faster. [Corrected 9 October: one task with a judged run on both sides was left out; on 169 tasks it is 59 against 55, 59% cheaper.]
Max effort adds little over high effort: on the 152 tasks where both have a judged run, it scored 60 against 58 at more than twice the cost. It also has a score for only 154 of the 200 tasks, most of the missing ones long, so its cost per task is understated.
Head-to-head on the same tasks
Twelve switches worth knowing about, each measured on the tasks both configurations ran. The arrow points from the configuration you would leave to the one you would pick; all cards share the same axes. Every paired comparison is listed further down.
How it was run
The task set
BU Bench V2.1 is a set of 200 long, multi-step web tasks, each scored against a weighted rubric. SLICC ran release v2.1.1. The runner decrypts the tasks in memory only, so no task text is stored in SLICC's repository.
The harness
Each task ran on a fresh SLICC instance: SLICC in Chrome on a cloud VM, with a wiped browser profile. The runner drives it from outside with the slicc command-line tool, so the agent does the task the way it would for a user. For every task the runner opens a new session with the agent's memories erased, sets the model and thinking effort, and sends the task text with a fixed instruction: do not ask clarifying questions, and end with a final answer line. It takes screenshots while the agent works and waits for the agent to finish, then records the cost and exports the transcript.
All models ran on AWS Bedrock, and every run used SLICC's built-in skills. The two intent rows also used the intent skill, at commit 9d10326 of ai-ecoverse/skills#473 before it merged, with kev-4b-vision as the small model on a GPU. Cost per task is SLICC's recorded spend for the agent and all its sub-agents, at list prices from pi-mono's published model list, or from models.dev where that list has no price. For Claude Haiku 5.5, which ran with pi 1.1.0, a request with more than 100k input tokens is billed at 5 times the base rate. Time per task is how long the prompt took. From the full runs on, the runner also waits until every sub-agent is idle, plus a 2-minute quiet period, and that wait is part of the time. The screening round and the intent runs ran without it, so times are not fully comparable between rows.
Default effort is the model's default, adaptive thinking. Claude Opus 5.5 on Bedrock cannot turn thinking off, and low effort is the lowest level that works. Max effort is thinking level xhigh combined with effort max. For GPT-6.1 Sol, low, high and max effort are SLICC's thinking levels low, high and max. For Grok 4.7, low and high effort are SLICC's thinking levels low and high.
Limits per run
- One hour per run, as the benchmark specifies.
- A cost cap of $10 per run. The screening batch with Claude Fable 5.1, GPT-6 Astra and Kimi K3 ran with a $30 cap. Most Claude Haiku 5.5 runs ran with a $5 cap; two of its max-effort runs ended near $5, and both also reached the one-hour limit.
- A run that reaches a limit is stopped and judged on the work it had done.
- The limits were not always enforced. Until a SLICC fix on 27 September, stopping a run did not stop the agent, so some runs in the screening round kept working and spending without being measured. Their recorded costs are probably too low. Some recorded runs go past the limits, up to 4,292 seconds and, for one GPT-6 Astra run, $44.879. 54 of the Grok 4.7 runs in the table, from 9 October, ended above the $10 cap, the highest at $18.035; the cause is not confirmed.
The judge
The judge is GPT-5.6 Luna on Bedrock. When a Luna verdict is still invalid after repair turns, GPT-5.6 Sol judges the run instead. This page does not break down how many runs each judge scored. The judge follows Browser Use's findings method: it marks each rubric item as met, violated or not assessable, and code computes the score as the met weight over the total weight. A run scores 1 when every item is met and 0 when none is. The judge prompt is read from Browser Use's repository at the pinned release, and the agent's own reasoning is never sent to the judge. Both judge models are from OpenAI, the vendor of the GPT-6 models under test.
Caveats
- The table has no confidence intervals. Each full-run row counts one run per task. Where Claude Opus 5.5 ran the same task three to five times across the screening round and the full run, its best and worst scores on that task were about 31 points apart on average.
- Errored runs are left out of the scores. 91 runs ended in an error and 13 more have no verdict from the judge. None of them count toward a score; the 13 without a verdict still count toward time and cost. Most errors come from the harness, for example an agent that kept working after the stop signal, a transcript export that timed out or a dropped connection. The errors are not spread evenly. 13 of the 190 runs of Claude Opus 5.5 at max effort errored, 12 of them because the agent ran past its limits or kept working after the stop, and 1 more has no verdict. Counting those 14 runs as a score of 0 moves its mean from 0.535 to 0.495. 45 of the 200 runs of GPT-6.1 Sol at max effort errored, most of them on long tasks.
- Nine of the 200 tasks have no run from Claude Opus 5.5, Claude Sonnet 5.5 or GPT-6 Luna: bu2-135, bu2-136, bu2-155, bu2-156, bu2-175, bu2-176, bu2-182, bu2-195 and bu2-196. They were part of the earlier full-run plan, and the cause of the gap is not confirmed. Claude Sonnet 5.5 at both efforts and GPT-6 Luna also have no run for 3 more tasks each, and Claude Opus 5.5 at max effort for 1.
- Results pool several SLICC versions. Releases shipped while the benchmark ran, so the runs span eight versions, from 6.195.0 to 6.220.1, the intent runs ran on 6.238.2 to 6.238.6, a few days later, Claude Haiku 5.5 and the 7 re-run GPT-6 Luna tasks on 6.242.1, and Grok 4.7 on 6.245.0. The screening round ran before the full runs and before several harness fixes landed. Claude Haiku 5.5 and Grok 4.7 ran with pi 1.1.0, the other runs with pi 0.99.x.
- Scores drift between days. Claude Haiku 5.5 and Grok 4.7 ran 7 to 11 days after the full runs without intent they are compared with. On 40 of the tasks, Claude Sonnet 5.5 at low effort scored 36.6 on 8 October against 37.0 on 28 and 29 September (difference -0.5 points, 95% interval -4.1 to +3.3), but other configurations moved by up to about 6 points between days. Score gaps of a few points between Claude Haiku 5.5 or Grok 4.7 and the older rows are within that range.
- Seven GPT-6 Luna runs from 28 September had started sub-agents on Claude Sonnet 4.6, which made up 85% of the row's spend. They were re-run on 9 October with GPT-6 Luna only, and the table counts the re-runs.
- Until a SLICC fix on 28 September, any GPT-6 turn that attached a screenshot the way SLICC sent it failed on Bedrock. GPT-6 Sol and GPT-6 Astra ran only before the fix, and so did part of the GPT-6 Luna runs. The affected runs have not been counted, and the GPT-6 scores may be lower than they would be on the fixed build. GPT-6.1 Sol ran after the fix, so the comparison between the two Sol versions also measures the fix.
- On 30 September, Bedrock returned server errors for hours while the GPT-6.1 Sol runs at high and max effort were going. A SLICC runner fix that day records a run whose last turn died on a provider error as an error instead of judging it. The affected tasks were run again on 30 September and 1 October, and runs that still died count as errors. The earlier runs on this page used the runner from before the fix, which judged those runs anyway. A check of their records found that 0 to 3% of runs per configuration died this way. For GPT-6 Luna the share was 5%.
- The judge gets less evidence than in Browser Use's own runner. It receives at most 10 screenshots, where Browser Use sends up to 50. Deliverable files such as a CSV or JSON output are not passed to it as files; it sees their contents only where they appear in the transcript. SLICC also leaves the judge's reasoning effort unset, where Browser Use's runner uses xhigh.
- These are SLICC's own runs, with SLICC's harness, browser setup and judge configuration. Browser Use's published V2 charts use an earlier 60-task set, so their scores are not comparable with the scores on this page.
All head-to-head comparisons
Each comparison pairs two configurations task by task and counts a pair only when both runs were judged; n is the number of pairs. Each covers its own set of tasks, so a score here can differ from the table above. Every comparison with n below 50 involves a configuration from the screening round. The comparisons with intent are in their own section, except two against Claude Haiku 5.5 below.
Against the older version
- Claude Sonnet 5.5 against Claude Sonnet 5: 43.5 against 37.4 (+16.2%), 77% cheaper, 63% faster. n = 14.
- GPT-6.1 Sol against GPT-6 Sol: 38 against 37 (+1.8%), 53% cheaper, 24% faster. n = 15.
Against the model at the other provider
- Claude Fable 5.1 against GPT-6 Astra: 44 against 43 (+2.3%), 38% cheaper, 240% slower. n = 19.
- Claude Opus 5.5 against GPT-6 Sol: 55 against 33 (+69.2%), 61% cheaper, 27% slower. n = 31.
- Claude Opus 5.5 against GPT-6.1 Sol: 51 against 53 (-4.1%), the same cost to within 1%, 69% slower. n = 189.
- Claude Sonnet 5.5 against GPT-6.1 Sol: 51 against 53 (-4.5%), 19% cheaper, 67% slower. n = 182.
- GPT-6.1 Sol at high effort against Claude Opus 5.5 at max effort: 59 against 55 (+8.5%), 59% cheaper, 45% faster. n = 169.
One tier up at the same provider
- Claude Opus 5.5 against Claude Sonnet 5.5: 50.4 against 50.5 (-0.2%), 22% more expensive, 2% slower. n = 183.
- Claude Fable 5.1 against Claude Opus 5.5: 46 against 58 (-20.1%), 349% more expensive, 29% slower. n = 18.
- Claude Opus 5.5 against Claude Sonnet 5: 56 against 37 (+48.7%), 77% cheaper, 52% faster. n = 14.
- GPT-6 Astra against GPT-6 Sol: 40 against 36 (+9.9%), 142% more expensive, 48% faster. n = 16.
Thinking effort, against the same model at its default
- Claude Opus 5.5 at low effort: 37 against 51 (-27.9%), 72% cheaper, 61% faster. n = 189.
- Claude Opus 5.5 at max effort: 53 against 51 (+4.4%), 286% more expensive, 90% slower. n = 176.
- Claude Sonnet 5.5 at low effort: 34 against 51 (-32.2%), 86% cheaper, 76% faster. n = 180.
- GPT-6.1 Sol at low effort: 45 against 53 (-15.2%), 46% cheaper, 13% faster. n = 195.
- GPT-6.1 Sol at high effort: 60 against 53 (+12.4%), 64% more expensive, 85% slower. n = 192.
- GPT-6.1 Sol at max effort: 60 against 52 (+15.9%), 246% more expensive, 150% slower. n = 154.
Claude Haiku 5.5
Claude Haiku 5.5 ran on 8 and 9 October, after every other row, so score gaps of a few points are within day-to-day drift. Its time includes the 2-minute settle wait; the intent runs' time does not.
- Claude Haiku 5.5 at max effort against Claude Sonnet 5.5: 49 against 51 (-2.7%), 21% cheaper, 45% slower. n = 182.
- Claude Haiku 5.5 at medium effort against Claude Sonnet 5.5 at low effort: 38 against 35 (+10.2%), 43% cheaper, 151% slower. n = 184.
- Claude Haiku 5.5 at medium effort against Claude Opus 5.5 at low effort: 38 against 37 (+3.5%), 77% cheaper, 50% slower. n = 189.
- Claude Haiku 5.5 at max effort against GPT-6.1 Sol at low effort: 49 against 45 (+9.6%), 23% more expensive, 185% slower. n = 194.
- Claude Haiku 5.5 at medium effort against GPT-6.1 Sol at low effort with intent: 39 against 40 (-4.4%), 55% cheaper, 107% slower, 74% without the settle wait. n = 198.
- Claude Haiku 5.5 at low effort against Claude Sonnet 5.5 at low effort with intent: 32 against 29 (+8.4%), 82% cheaper, 171% slower, 92% without the settle wait. n = 199.
- Claude Haiku 5.5 at low effort against GPT-6 Luna: 32 against 28 (+14.6%), 70% cheaper, 33% slower. n = 187.
- Claude Haiku 5.5 at max effort against Claude Opus 5.5: 48 against 51 (-4.3%), 37% cheaper, 41% slower. n = 188.
- Claude Haiku 5.5 at medium effort against Claude Haiku 5.5 at low effort: 39 against 32 (+20.8%), 424% more expensive, 81% slower. n = 198.
- Claude Haiku 5.5 at max effort against Claude Haiku 5.5 at medium effort: 48 against 39 (+25.6%), 920% more expensive, 144% slower. n = 196.
Grok 4.7
Grok 4.7 ran on 9 October, so score gaps of a few points against the older rows are within day-to-day drift.
- Claude Haiku 5.5 at max effort against Grok 4.7 at high effort: 49 against 43 (+12.0%), 75% cheaper, 63% slower. n = 197.
- GPT-6.1 Sol at low effort against Grok 4.7 at high effort: 45 against 43 (+4.1%), 79% cheaper, 40% faster. n = 195.
- Claude Sonnet 5.5 against Grok 4.7 at high effort: 50 against 43 (+18.1%), 68% cheaper, 12% slower. n = 182.
- Grok 4.7 at high effort against Grok 4.7 at low effort: 43 against 40 (+7.0%), 4% cheaper, 1% slower. n = 198.
- Claude Opus 5.5 against Grok 4.7 at high effort: 50 against 43 (+17.9%), 60% cheaper, 15% slower. n = 189.
- GPT-6.1 Sol against Grok 4.7 at high effort: 53 against 43 (+23.4%), 61% cheaper, 32% faster. n = 198.
Attribution and data
The tasks, rubrics and judging method are from BU Bench V2.1 by Browser Use: github.com/browser-use/benchmark, release v2.1.1, commit af6c7f7, dataset SHA-256 0c014b056192e261b06f04ed6da6dcf37f2135150cf58c87ea839bd4e150e5c3. Browser Use did not run, review or endorse these results. At Browser Use's request, this page reproduces no task text, rubric or judge prompt.
The scores are SLICC's own measurements, taken from SLICC's benchmark dashboard of 29 September 2026 and, for GPT-6.1 Sol, from the reports of its benchmark runs of 30 September and 1 October 2026. The intent rows and comparisons come from the run report of 8 October 2026, recomputed from the intent runs of 4 to 6 October and the earlier runs they are compared with. The Claude Haiku 5.5 rows and comparisons, the corrected GPT-6 Luna row and the one-run-per-task row for Claude Opus 5.5 at default effort come from the run report of 9 October 2026, recomputed from the run records of 28 September to 9 October. The Grok 4.7 rows and comparisons come from a second run report of 9 October 2026, recomputed from the Grok 4.7 runs of that day. The BU Bench V2.1 run records behind them are not public at the time of writing. The public Hugging Face dataset ai-ecoverse/slicc-bench holds earlier runs on BU Bench V1 only and is not the source of these numbers. The benchmark runner is open source, in the packages/bench folder of the SLICC repository.