Trawl MCP, CLI or API: Which One to Use for Your Job
Which Trawl surface fits your job? A table by task, plus the limits that decide it: what an API key can do, how long calls block, and which plans gate what.
Use the web app to build a scraper by hand, MCP when an AI agent should drive Trawl, the CLI for scripts and CI, and the REST API from your own backend or from n8n. All four talk to the same scrapers, so the choice rarely depends on what they can read. It depends on three things: which credential you hold, how long a call blocks, and which settings a surface can change.
This guide maps common jobs to a surface, then shows the limits that decide it, with real output from a test account. Every command below ran against a Trawl test environment with CLI version 4.0.4 (the current latest on npm when we wrote this). The scraper and run ids are from that test account, so swap in your own.
Pick by job
| The job | Use | The limit to know |
|---|---|---|
| Build one scraper and look at the result | Web app | Sign in with email and password. Each wizard session costs compute. |
| Let a coding agent create, run and read scrapers | MCP | Bearer token only. 11 tools, none that edits or deletes. |
| Run scrapers from CI or cron and use the rows in a shell | CLI | An API key works for most commands. A few need a login. |
| Create scrapers from your own code, with a schedule set at creation | REST | The create call is silent until the scraper is built. |
| Call Trawl from n8n | REST, with the HTTP Request node | There is no Trawl node on npm. |
| Change a schedule, edit or delete a scraper | Web app, or the CLI after trawl login | An API key gets a 401 on these. |
| Create or edit a webhook | Web app | Growth and Pro plans only. |
If one line of the table matches your job, you can stop here. The rest explains why each limit exists, so you can check it against your own setup.
The same 37 rows, read three ways
Reading data is the one job all of the programmatic surfaces do the same way, and it costs no compute: it returns the last stored run instead of starting a new one. We used a scraper that lists every article of the Trawl blog, 37 of them on the day we ran it, built the way the first tutorial builds one.
From the CLI, with TRAWL_API_KEY set to a key made in the web app:
$ trawl data 6adcaa98c02095a8796457b3
Last run data:
Items: 37
First item keys: date, link, tag, title
Use --json for full output.From MCP, a plain JSON-RPC call to the trawl_scrap_latest_result tool. Here the reply is cut down to its status and row count with jq:
curl -s -X POST https://api.trawl.me/api/mcp \
-H "Authorization: Bearer $TRAWL_API_KEY" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":
{"name":"trawl_scrap_latest_result",
"arguments":{"scrapId":"6adcaa98c02095a8796457b3"}}}' \
| jq -c '.result.structuredContent
| {status, statusDetail, rows: (.result.data | length)}'{"status":true,"statusDetail":"success","rows":37}From REST, with the id of the run that trawl_scrap_latest_result or GET /api/scraps/:id gives you. The rows sit in a JSON string inside the run record:
curl -s https://api.trawl.me/api/runs/6adcaa98c02095a879645b22 \
-H "Authorization: Bearer $TRAWL_API_KEY" \
| jq -c '.data.data | fromjson | .data
| length, (.[0] | {date, tag, title})'37
{"date":"2026-10-19","tag":"guides",
"title":"How to Track Expedia Prices Without Getting Blocked"}Same 37 rows each time: we hashed the three row lists and the hashes matched. The difference is the wrapper: the CLI gives a summary (add --json for every row), MCP gives the whole run record, and REST gives the run record with the rows as a string. If you only want the rows in a pipeline, the CLI with --json is the shortest path.
Limit 1: what your credential can do
A Trawl API key is created in the web app, under Developers then API Keys. You give it a name, an expiry date and, if you want, a list of scrapers it may touch. Keys are available on every plan, including Free. A key made in that form can do every action. To narrow the actions, create the key over the API with an explicit scopes list.
Four scopes matter for what this guide covers:
| Scope | What it lets the key do |
|---|---|
trawl:scraps:read | List scrapers and read one, with its latest result |
trawl:history:read | Read run history and single runs |
trawl:scraps:run | Start a run (trawl trigger, the MCP run tool) |
trawl:scraps:create | Create a scraper (trawl create, the MCP create tool) |
We made a key with only the first two and tried the actions it should not have. The CLI says so plainly (it also warns that this scraper has no schedule, which is true of ours, so we left that line out):
$ trawl trigger 6adcaa98c02095a8796457b3
✗ 403 Forbidden: API key missing required scope(s): trawl:scraps:run
$ trawl create https://blog.trawl.me --prompt "scope check only"
✗ 403 Forbidden: API key missing required scope(s): trawl:scraps:createMCP answers the same question with a JSON-RPC error. The call still returns HTTP 200, so read isError and the code, not the status line:
{
"result": {
"content": [{"type": "text", "text": "{\"code\":-32001,\"message\":
\"API key is missing the required scope: trawl:scraps:run\"}"}],
"isError": true
},
"jsonrpc": "2.0", "id": 4
}A read-only key is a good default for a dashboard or an agent that should only look. It still passes trawl_whoami and every read tool, and it lists scrapers and fetches results as usual.
Some actions are not open to a key at all. With a key, trawl whoami refuses and tells you it needs a session (a login), and trawl scraps update comes back with "The API key was rejected (401)". The same goes for scraps delete, scrap account and session commands, scraps banner, scraps snapshot and the standalone scraps watch. Run trawl login first and the same command works. Changing a schedule from the terminal looks like this:
$ trawl scraps update 6adcaa98c02095a8796457b3 --cron "0 8 * * 1"
cron: 0 8 * * 1A login lasts seven days, which is why unattended jobs use a key and interactive changes use a login. Over REST the picture is the same: PUT /api/scraps/:id with a key returns 401, and webhook management also needs a login rather than a key.
MCP takes a Bearer token and nothing else
The Trawl MCP endpoint accepts a Bearer token, either an API key or a login token. Without one it returns 401 with WWW-Authenticate: Bearer, and there is no OAuth sign-in step and no mode without authentication. So a client has to let you set a static Authorization header. Claude Code does, and our MCP guide covers connecting it.
A chat app whose connector screen only offers a sign-in button, or a bare URL, cannot connect to Trawl today. We did not test any chat app here. Which AI chat can use Trawl has the tested results. Otherwise look for a header or token field in yours before you plan around it. If it has none, use the CLI or REST from the same account instead.
To see exactly what an agent gets, ask the server. A tools/list call on the endpoint above returns 11 tools:
trawl_list_scraps trawl_get_scrap trawl_trigger_scrap_run
trawl_scrap_latest_result trawl_get_scrap_runs trawl_get_run
trawl_diagnose_scrap trawl_get_run_snapshot trawl_whoami
trawl_health_ping trawl_create_scrapThat is create, run, read and diagnose. There is no tool to change a schedule, edit or delete a scraper. For an agent that is often a feature: it can build and watch scrapers, and it cannot remove one.
Limit 2: how long a call blocks
Building a scraper takes real time, because the AI writes the script and tests it on the live page. The surfaces handle the wait differently, and that matters more than any other difference when you wire Trawl into something with a timeout.
- MCP blocks until the work is done.
trawl_trigger_scrap_runusually takes 30 to 250 seconds.trawl_create_scrapusually takes 60 to 400 seconds, and the API closes the connection after about 12 minutes while the scraper is still being created. Set your client's tool timeout above that, and list your scrapers before you retry a create, because it may already exist. - CLI:
trawl runblocks until the run finishes.trawl triggerreturns at once and the run continues in the background.trawl createprints one line per stage while it works. - REST: the create call (
POST /api/ai/wizard) sends nothing until the scraper is built, so a plain Nodefetchgives up after about five minutes unless you lift its timeouts. The run route accepts?wait=falseand returns a queued status instead of the rows.
On a busy test server, one MCP create from Claude Code took about six and a half minutes. Plan for the slow end.
Limit 3: what a new scraper gets by default
Whatever door you use to create a scraper, it starts with the same safety net: a weekly run, auto-fix on, and an email digest to the account address. The details differ slightly:
- Web app: weekly, Monday 07:00 in your browser's timezone. You can switch to daily on the last step.
- CLI, MCP and REST: weekly, Monday 07:00 UTC. Reading a scraper created from the CLI through the REST API shows
0 7 * * 1with the timezoneUTC, auto-fix on, and the digest enabled for the account email.
To skip the digest, pass --no-digest to trawl create, or "digest": {"enabled": false} in the REST body. The MCP create tool has no digest argument. Of the three, only REST takes a cron field at creation. From the CLI, create first and then run trawl scraps update --cron after a login.
Compute follows the work, not the surface. The free plan comes with 10,000 compute, no card required. Building a scraper is the expensive part, because the AI writes the script: about 2,700 compute on a simple page in our test, more on large or protected pages. After that, each run costs about 50 to 100 compute on a simple page. The free compute covers a few scrapers like this one (about three on simple pages) and their weekly runs. Reading stored data costs nothing.
Limit 4: plan gates
Two developer features need a Growth or Pro plan: creating or editing a webhook, and the live JSON endpoint GET /api/scraps/:id/data. A free account gets this from the endpoint:
{
"type": "error", "message": "Forbidden", "code": 403,
"errorCode": "PLAN_REQUIRED",
"requiredPlans": ["growth", "pro"], "currentPlan": "free"
}This is the reason the read example above went through the run record: it works on every plan. Everything else in this guide works on Free, limited by your compute.
Deciding in five steps
- Name the verb. Build, run, read, change or delete. Reading and running are open to every surface. Changing and deleting are not.
- Name the caller. A person at a screen, an agent in a coding session, a script on a schedule, or your own service. The person gets the web app, the agent gets MCP, the script gets the CLI, the service gets REST.
- Pick the credential. A key for anything that runs unattended, a login for the few commands that edit or delete. Narrow the key with
scopesif the caller only needs to read. - Check the wait. If the caller has a timeout under 12 minutes, avoid blocking calls: use
trawl trigger, or REST with?wait=false, then read the data later. - Try it with a read-only key first. Run
trawl listandtrawl datawith the key, and add the write scopes only when the first two work.
To confirm what your installed CLI offers, trawl spec --json prints its whole command tree, and trawl skills install adds five skills that teach Claude Code to use them.
Want to test a surface with your own page? Start free on Trawl, 10,000 free compute and no card.
Where to go next
Pick the surface that fits and the rest follow the same account. If you are starting from zero, the first tutorial builds a scraper in the web app, and the try page shows the shape of the data before you sign up. When a scraper breaks later, self-healing scrapers explains how the repair works. For the compute side of all this, see what Trawl 3.3 changed.
Ready to wire one up? Create a Trawl account, make a key, and run trawl list before anything else.
Disclaimer: Trawl provides scraping infrastructure. Users are responsible for ensuring their use complies with applicable laws and website terms of service. This article is for educational purposes only.
Written by Pierre | November 2026