From aa32e3c339081233abae922b4ef8b5a783be1fff Mon Sep 17 00:00:00 2001 From: alitariksahin Date: Tue, 18 Aug 2026 14:37:06 +0300 Subject: [PATCH 1/6] feat: add captcha solving logic --- box/overall/browser/ai-actions.mdx | 28 ++++++++++++++++++++++++++++ box/overall/browser/cookbook.mdx | 26 +++++++------------------- 2 files changed, 35 insertions(+), 19 deletions(-) diff --git a/box/overall/browser/ai-actions.mdx b/box/overall/browser/ai-actions.mdx index 597861d1..0c41945f 100644 --- a/box/overall/browser/ai-actions.mdx +++ b/box/overall/browser/ai-actions.mdx @@ -84,6 +84,32 @@ tab.act(action) Observe narrowly (or check the element) before relying on a fixed index like `elements[0]`. Cache the returned action (in your own store or on the box filesystem) and replay it across pages or runs. This is the built-in path for turning an AI-discovered step into a fast, repeatable one. The action must carry a resolved `selector`: `act()` throws if it is missing (an `observe()` element it could not resolve). The replay form runs no model, so a `model` override does not apply. If the page changes and the selector no longer matches, `observe()` again to re-resolve. +### Built-in captcha solving (beta) + +An `act()` can transparently solve a reCAPTCHA v2 checkbox that blocks the page before it runs your instruction. It is opt-in per call and off by default: set `solveCaptchas: true` (TypeScript) or `solve_captchas=True` (Python). It also applies to the replay form. + + +```typescript box.ts +const action = await tab.act("submit the form", { solveCaptchas: true }) + +// Present only when solving was requested +console.log(action.captcha) // { attempted, solved, skipped? } +``` + +```python box.py +action = tab.act("submit the form", solve_captchas=True) + +# Present only when solving was requested +print(action.captcha) # attempted, solved, skipped +``` + + +The `captcha` field reports whether solving was `attempted`, whether it `solved`, and a `skipped` reason when it did not run (for example `"unconfigured"` or `"monthly_cap"`). Solving is best-effort and never fails your action: if it cannot solve, the instruction still runs. + + + Built-in captcha solving is in beta. It covers the reCAPTCHA v2 checkbox only and is best-effort. + + ## Which one to use | Method | Does | Best for | @@ -92,3 +118,5 @@ Observe narrowly (or check the element) before relying on a fixed index like `el | `act` | Executes one action (natural language, metered; or a pre-resolved action, no LLM) | Flows where your code decides each step, or replaying a resolved action | To turn a single AI-resolved step into a no-LLM one, replay an `observe()` result through `act()` (see [Replay an action without an LLM](#replay-an-action-without-an-llm)). For fully scripted control with no LLM anywhere in the loop, [connect over CDP](/box/overall/browser/connect) with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see [Live View](/box/overall/browser/live-view) and [Recordings](/box/overall/browser/recordings). + +For goal-driven, multi-step browsing (the replacement for the removed `tab.run()`), see [Agentic browsing](/box/overall/browser/cookbook#agentic-browsing) in the cookbook. diff --git a/box/overall/browser/cookbook.mdx b/box/overall/browser/cookbook.mdx index e66027a6..0aeeaa47 100644 --- a/box/overall/browser/cookbook.mdx +++ b/box/overall/browser/cookbook.mdx @@ -7,29 +7,21 @@ description: "Runnable browser examples grouped by use case. Each one is self-co Every example is a single self-contained file in the [Box SDK repo](https://github.com/upstash/box/tree/main/packages/sdk/examples/browser). All of them need `UPSTASH_BOX_API_KEY`. Examples marked **AI** use metered - browser AI (`run`, `act`, `extract`) and need a model provider key on the box + browser AI (`act`, `extract`, `observe`) and need a model provider key on the box or account. Everything else runs with the Box key alone. -## Agents +## Agentic browsing -Goal-driven browsing with `tab.run()`. +Goal-driven browsing: hand a goal to the box's agent, or drive your own loop. - - Constrained search with a fallback category. The agent evaluates results, rejects with reasons, and switches on its own. **AI** + + Hand a goal to `box.agent.run`; the box's agent drives the browser through the chrome-devtools MCP and writes results to the box (needs an agent key; bills coding-agent tokens). **Agent** - - The same task via Playwright, `act` plus `extract`, and `run`. Pick your autonomy level by token cost. **AI** - - - - The search again with live view, session recording, a decision log, and token accounting. **AI** - - - - One prompt and one schema across three differently structured sites. **AI** + + A live `observe` then `act(action)` then `extract` loop you own, with `extract` as the stop check. **AI** @@ -90,10 +82,6 @@ The box as a test environment. The box hosts the app under test and browses it on its own localhost. - - Agent-driven smoke flow, cross-checked by deterministic DOM assertions, recorded on video. **AI** - - Pixelmatch diffs against baselines stored on the box. From c714a8b54d23b80e148652481492442d5deb55c7 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 18 Aug 2026 11:37:45 +0000 Subject: [PATCH 2/6] chore(llms): regenerate llms.txt and llms-full.txt --- llms-full.txt | 54 +++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 19 deletions(-) diff --git a/llms-full.txt b/llms-full.txt index ee1635f1..27831337 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -2860,6 +2860,32 @@ tab.act(action) Observe narrowly (or check the element) before relying on a fixed index like `elements[0]`. Cache the returned action (in your own store or on the box filesystem) and replay it across pages or runs. This is the built-in path for turning an AI-discovered step into a fast, repeatable one. The action must carry a resolved `selector`: `act()` throws if it is missing (an `observe()` element it could not resolve). The replay form runs no model, so a `model` override does not apply. If the page changes and the selector no longer matches, `observe()` again to re-resolve. +### Built-in captcha solving (beta) + +An `act()` can transparently solve a reCAPTCHA v2 checkbox that blocks the page before it runs your instruction. It is opt-in per call and off by default: set `solveCaptchas: true` (TypeScript) or `solve_captchas=True` (Python). It also applies to the replay form. + + +```typescript box.ts +const action = await tab.act("submit the form", { solveCaptchas: true }) + +// Present only when solving was requested +console.log(action.captcha) // { attempted, solved, skipped? } +``` + +```python box.py +action = tab.act("submit the form", solve_captchas=True) + +# Present only when solving was requested +print(action.captcha) # attempted, solved, skipped +``` + + +The `captcha` field reports whether solving was `attempted`, whether it `solved`, and a `skipped` reason when it did not run (for example `"unconfigured"` or `"monthly_cap"`). Solving is best-effort and never fails your action: if it cannot solve, the instruction still runs. + + + Built-in captcha solving is in beta. It covers the reCAPTCHA v2 checkbox only and is best-effort. + + ## Which one to use | Method | Does | Best for | @@ -2869,6 +2895,8 @@ Observe narrowly (or check the element) before relying on a fixed index like `el To turn a single AI-resolved step into a no-LLM one, replay an `observe()` result through `act()` (see [Replay an action without an LLM](#replay-an-action-without-an-llm)). For fully scripted control with no LLM anywhere in the loop, [connect over CDP](/docs/box/overall/browser/connect) with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see [Live View](/docs/box/overall/browser/live-view) and [Recordings](/docs/box/overall/browser/recordings). +For goal-driven, multi-step browsing (the replacement for the removed `tab.run()`), see [Agentic browsing](/docs/box/overall/browser/cookbook#agentic-browsing) in the cookbook. + # Connect over CDP Source: https://upstash.com/docs/box/overall/browser/connect @@ -2959,29 +2987,21 @@ Source: https://upstash.com/docs/box/overall/browser/cookbook Every example is a single self-contained file in the [Box SDK repo](https://github.com/upstash/box/tree/main/packages/sdk/examples/browser). All of them need `UPSTASH_BOX_API_KEY`. Examples marked **AI** use metered - browser AI (`run`, `act`, `extract`) and need a model provider key on the box + browser AI (`act`, `extract`, `observe`) and need a model provider key on the box or account. Everything else runs with the Box key alone. -## Agents +## Agentic browsing -Goal-driven browsing with `tab.run()`. +Goal-driven browsing: hand a goal to the box's agent, or drive your own loop. - - Constrained search with a fallback category. The agent evaluates results, rejects with reasons, and switches on its own. **AI** + + Hand a goal to `box.agent.run`; the box's agent drives the browser through the chrome-devtools MCP and writes results to the box (needs an agent key; bills coding-agent tokens). **Agent** - - The same task via Playwright, `act` plus `extract`, and `run`. Pick your autonomy level by token cost. **AI** - - - - The search again with live view, session recording, a decision log, and token accounting. **AI** - - - - One prompt and one schema across three differently structured sites. **AI** + + A live `observe` then `act(action)` then `extract` loop you own, with `extract` as the stop check. **AI** @@ -3042,10 +3062,6 @@ The box as a test environment. The box hosts the app under test and browses it on its own localhost. - - Agent-driven smoke flow, cross-checked by deterministic DOM assertions, recorded on video. **AI** - - Pixelmatch diffs against baselines stored on the box. From 55c164a2b37f653302808fee67ae699d34dd91af Mon Sep 17 00:00:00 2001 From: alitariksahin Date: Tue, 18 Aug 2026 19:15:58 +0300 Subject: [PATCH 3/6] chore: remove captcha --- box/overall/browser/ai-actions.mdx | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/box/overall/browser/ai-actions.mdx b/box/overall/browser/ai-actions.mdx index 0c41945f..18667359 100644 --- a/box/overall/browser/ai-actions.mdx +++ b/box/overall/browser/ai-actions.mdx @@ -84,32 +84,6 @@ tab.act(action) Observe narrowly (or check the element) before relying on a fixed index like `elements[0]`. Cache the returned action (in your own store or on the box filesystem) and replay it across pages or runs. This is the built-in path for turning an AI-discovered step into a fast, repeatable one. The action must carry a resolved `selector`: `act()` throws if it is missing (an `observe()` element it could not resolve). The replay form runs no model, so a `model` override does not apply. If the page changes and the selector no longer matches, `observe()` again to re-resolve. -### Built-in captcha solving (beta) - -An `act()` can transparently solve a reCAPTCHA v2 checkbox that blocks the page before it runs your instruction. It is opt-in per call and off by default: set `solveCaptchas: true` (TypeScript) or `solve_captchas=True` (Python). It also applies to the replay form. - - -```typescript box.ts -const action = await tab.act("submit the form", { solveCaptchas: true }) - -// Present only when solving was requested -console.log(action.captcha) // { attempted, solved, skipped? } -``` - -```python box.py -action = tab.act("submit the form", solve_captchas=True) - -# Present only when solving was requested -print(action.captcha) # attempted, solved, skipped -``` - - -The `captcha` field reports whether solving was `attempted`, whether it `solved`, and a `skipped` reason when it did not run (for example `"unconfigured"` or `"monthly_cap"`). Solving is best-effort and never fails your action: if it cannot solve, the instruction still runs. - - - Built-in captcha solving is in beta. It covers the reCAPTCHA v2 checkbox only and is best-effort. - - ## Which one to use | Method | Does | Best for | From 659bc6a1171d53536dad9ac0a4f14ae5b19aaff0 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:17:15 +0000 Subject: [PATCH 4/6] chore(llms): regenerate llms.txt and llms-full.txt --- llms-full.txt | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/llms-full.txt b/llms-full.txt index 27831337..4e5cb367 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -2860,32 +2860,6 @@ tab.act(action) Observe narrowly (or check the element) before relying on a fixed index like `elements[0]`. Cache the returned action (in your own store or on the box filesystem) and replay it across pages or runs. This is the built-in path for turning an AI-discovered step into a fast, repeatable one. The action must carry a resolved `selector`: `act()` throws if it is missing (an `observe()` element it could not resolve). The replay form runs no model, so a `model` override does not apply. If the page changes and the selector no longer matches, `observe()` again to re-resolve. -### Built-in captcha solving (beta) - -An `act()` can transparently solve a reCAPTCHA v2 checkbox that blocks the page before it runs your instruction. It is opt-in per call and off by default: set `solveCaptchas: true` (TypeScript) or `solve_captchas=True` (Python). It also applies to the replay form. - - -```typescript box.ts -const action = await tab.act("submit the form", { solveCaptchas: true }) - -// Present only when solving was requested -console.log(action.captcha) // { attempted, solved, skipped? } -``` - -```python box.py -action = tab.act("submit the form", solve_captchas=True) - -# Present only when solving was requested -print(action.captcha) # attempted, solved, skipped -``` - - -The `captcha` field reports whether solving was `attempted`, whether it `solved`, and a `skipped` reason when it did not run (for example `"unconfigured"` or `"monthly_cap"`). Solving is best-effort and never fails your action: if it cannot solve, the instruction still runs. - - - Built-in captcha solving is in beta. It covers the reCAPTCHA v2 checkbox only and is best-effort. - - ## Which one to use | Method | Does | Best for | From e7abddd22ad5e3539dd47ac264bbe0963d0521f6 Mon Sep 17 00:00:00 2001 From: alitariksahin Date: Tue, 18 Aug 2026 19:20:59 +0300 Subject: [PATCH 5/6] chore: remove unnecessary run note --- box/overall/browser/ai-actions.mdx | 2 -- 1 file changed, 2 deletions(-) diff --git a/box/overall/browser/ai-actions.mdx b/box/overall/browser/ai-actions.mdx index 18667359..597861d1 100644 --- a/box/overall/browser/ai-actions.mdx +++ b/box/overall/browser/ai-actions.mdx @@ -92,5 +92,3 @@ Observe narrowly (or check the element) before relying on a fixed index like `el | `act` | Executes one action (natural language, metered; or a pre-resolved action, no LLM) | Flows where your code decides each step, or replaying a resolved action | To turn a single AI-resolved step into a no-LLM one, replay an `observe()` result through `act()` (see [Replay an action without an LLM](#replay-an-action-without-an-llm)). For fully scripted control with no LLM anywhere in the loop, [connect over CDP](/box/overall/browser/connect) with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see [Live View](/box/overall/browser/live-view) and [Recordings](/box/overall/browser/recordings). - -For goal-driven, multi-step browsing (the replacement for the removed `tab.run()`), see [Agentic browsing](/box/overall/browser/cookbook#agentic-browsing) in the cookbook. From 5a25888a9427e295b54ffcf88cf5d3a52e8c25fb Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 18 Aug 2026 16:21:39 +0000 Subject: [PATCH 6/6] chore(llms): regenerate llms.txt and llms-full.txt --- llms-full.txt | 2 -- 1 file changed, 2 deletions(-) diff --git a/llms-full.txt b/llms-full.txt index 4e5cb367..9ccee8ae 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -2869,8 +2869,6 @@ Observe narrowly (or check the element) before relying on a fixed index like `el To turn a single AI-resolved step into a no-LLM one, replay an `observe()` result through `act()` (see [Replay an action without an LLM](#replay-an-action-without-an-llm)). For fully scripted control with no LLM anywhere in the loop, [connect over CDP](/docs/box/overall/browser/connect) with Playwright or Puppeteer instead. Both drive the same tabs, so you can mix scripted steps with AI steps. To watch or replay what the agent did, see [Live View](/docs/box/overall/browser/live-view) and [Recordings](/docs/box/overall/browser/recordings). -For goal-driven, multi-step browsing (the replacement for the removed `tab.run()`), see [Agentic browsing](/docs/box/overall/browser/cookbook#agentic-browsing) in the cookbook. - # Connect over CDP Source: https://upstash.com/docs/box/overall/browser/connect