AI Guides › Workbench

ScrapeGraphAI: The Free Local Route and the Paid Hosted Route

By Nigel Guy · 7 min read

The usual way people meet ScrapeGraphAI is a short clip promising a free tool that "scrapes anything". So they paste two install lines into a terminal, hit a Python version error, connect the hosted version to Claude Code instead, and only later notice they are spending credits. There are really two products with one name. One is a free, open-source Python library you run on your own machine. The other is a paid cloud API that your AI assistant calls for you. Both are useful, but they cost and fail in different ways.

The words you need first

The routes compared

Route 1: the free local route, by hand

What you need: Python 3.12 or later (PyPI lists the latest release, 2.3.0, as needing at least 3.12), a terminal, a few gigabytes of free disk space, and ideally 8 GB of RAM or more. RAM needs vary by machine, so treat that as a guide, not a guarantee.

Step 1 — check Python. Run python3 --version (on Windows, python --version). If it shows 3.11 or older, install a current version from python.org before going any further.

Step 2 — make a project folder and a virtual environment. The package's own page recommends this.

mkdir scraper && cd scraper
python3 -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate

Step 3 — install the library and a browser.

pip install scrapegraphai
playwright install chromium

Plain playwright install downloads every browser Playwright supports. Adding chromium gets only the one you need.

Step 4 — install Ollama and fetch a model. Download Ollama from ollama.com, open it, then run:

ollama pull llama3.2

That fetches the default 3-billion-parameter version, which Ollama lists at about 2 GB. Keep the Ollama app running while you scrape.

Step 5 — write the script. Save the following as scrape.py. Fill in TARGET_URL, and replace the field list in WHAT_TO_EXTRACT with what you actually want.

import json
from scrapegraphai.graphs import SmartScraperGraph

TARGET_URL = "[PAGE_URL]"
WHAT_TO_EXTRACT = (
    "You are extracting data from one web page for a spreadsheet. "
    "Return a JSON object with a key 'items', holding a list. "
    "Each entry has these fields: [FIELD_1], [FIELD_2], [FIELD_3]. "
    "Copy values exactly as written on the page. "
    "If a field is missing for an entry, use null. Never guess or fill gaps. "
    "Skip navigation, adverts and footer links."
)

settings = {
    "llm": {
        "model": "ollama/llama3.2",
        "model_tokens": 8192,
        "format": "json",
    },
    "headless": True,
}

scraper = SmartScraperGraph(prompt=WHAT_TO_EXTRACT, source=TARGET_URL, config=settings)
result = scraper.run()

with open("result.json", "w", encoding="utf-8") as f:
    json.dump(result, f, indent=2, ensure_ascii=False)
print(json.dumps(result, indent=2, ensure_ascii=False))

Step 6 — run it. In the same activated terminal, run python scrape.py. The results print on screen and are saved to result.json.

Route 2: the hosted route in Claude Code, Cursor or Codex

This route skips Python entirely, but every request runs on ScrapeGraphAI's cloud and uses credits. Make an account at scrapegraphai.com first, then add the server for your tool:

To check it worked, ask the assistant to run the credits tool. If it shows your balance, the connection works. Then give it a bounded job. Fill in the bracketed parts:

You have the ScrapeGraphAI (sgai) tools. I need structured data from one page.

Page: [PAGE_URL]
Fields I want for each item: [FIELD_LIST]
Credit limit for this task: [MAX_CREDITS]

Steps:
1. Check my credit balance first and tell me what this task should cost before you spend anything.
2. Use one extraction call on that single page only. Do not crawl, search, set up monitors or turn on stealth mode unless I say so.
3. Return a table with one row per item and one column per field. Use "not on page" where a value is missing. Never estimate one.
4. Below the table, list any fields that were ambiguous and how many items you found.

If the page URL or the field list is missing or unclear, ask me before calling any tool.
Before you answer, check that every value in the table came from the tool result and not from your own knowledge.

When it goes wrong

How to choose

• Choose local if you want it free, are scraping a handful of pages, or the content is something you'd rather not send to a third party.
• Choose hosted if you already work in Claude Code, Cursor or Codex, need it working in five minutes, and are happy to pay per request after the free credits.

What to skip

• The SearchGraph and multi-page graphs on day one. Get one page right first.
• Stealth mode on the hosted route. It doubles the credit cost, and if a site is actively blocking you, take that as a signal to stop.
• Monitors and crawls until you have checked a single-page result by hand.
• "Scrapes anything" claims. Logged-in pages, paywalls and sites that block bots are out of scope, and they should be.

Guardrails

• Check before you trust. Compare at least five rows against the live page before you use the data. Language models can produce tidy, confident, wrong rows.
• Read the site's terms and robots.txt. Some sites forbid automated collection. A free tool doesn't give you permission.
• Personal data is regulated. If the page holds names, emails or anything else that identifies people, UK GDPR applies to what you collect and keep. Take advice before building a list of people.
• Prices move. The plan names, credit costs and dollar prices above are from ScrapeGraphAI's site at time of writing. Check the pricing page before you pay.

Sources

• ScrapeGraphAI GitHub repository
• scrapegraphai on PyPI
• ScrapeGraphAI MCP server introduction
• ScrapeGraphAI MCP: Claude Code
• ScrapeGraphAI MCP: Cursor
• ScrapeGraphAI MCP: Codex
• ScrapeGraphAI pricing
• Ollama: Llama 3.2

All 751 AI guides · JulieMango plans from £17/mo