The Hidden Cost of HTML in AI Agents

Your AI Agent Has a Token Problem

Every time your AI agent reads a web page, it processes thousands of tokens of noise: navigation menus, footer links, ad scripts, tracking pixels, cookie banners, social sharing buttons, and inline CSS. None of this helps your agent understand the content.

Here's a real example. A typical blog post on a popular tech site:

That's 13,000 wasted tokens on a single page. At Claude Sonnet 4 pricing ($3/M input tokens), those wasted tokens cost $0.039 per page. Doesn't sound like much until you do the math at scale.

The Math Gets Ugly Fast

Consider a typical AI agent workflow that processes 100 pages per task:

That's $1,170 to $23,400 per month in wasted tokens. Just from feeding HTML instead of Markdown.

And this doesn't account for the second-order effects:

• Slower responses. More tokens means longer processing time. Your agent takes 3-4x longer to "read" a page.
• Context window pollution. Noise tokens push useful content out of the context window. Your agent literally forgets important information because its context is full of <div class="ad-wrapper">.
• Lower quality outputs. LLMs perform worse when signal-to-noise ratio is low. Navigation links and cookie consent text confuse the model about what matters.

The Cloudflare Gap

Cloudflare launched "Markdown for Agents" in February 2026. When an AI agent sends Accept: text/markdown, Cloudflare converts HTML to Markdown at the edge. Free for Pro/Business/Enterprise customers.

This is great for the ~20% of the web behind Cloudflare on paid plans. But the other 80% of the web still serves raw HTML. Your agent needs to read documentation on GitHub, articles on Medium, research papers, news sites, SaaS landing pages, and thousands of other sites that don't use Cloudflare's feature.

The Fix: Convert Once, Cache, Reuse

StripFeed solves this with a simple API. Send any URL, get clean Markdown:

curl "https://www.stripfeed.dev/api/v1/fetch?url=https://example.com/article" \
  -H "Authorization: Bearer sf_live_your_key"

The response headers tell you exactly what you saved:

X-StripFeed-Tokens: 3,150
X-StripFeed-Original-Tokens: 16,180
X-StripFeed-Savings-Percent: 80.5
X-StripFeed-Cache: MISS

On the second request for the same URL, you get a cache hit with zero processing time:

X-StripFeed-Cache: HIT
X-StripFeed-Fetch-Ms: 2

The cache is configurable: 1 hour by default, up to 24 hours for Pro users. For content that doesn't change frequently (documentation, blog posts, reference pages), this means your agent reads it once and every subsequent access is essentially free.

Token Counting with Cost Tracking

StripFeed counts tokens using the GPT-4o tokenizer (via js-tiktoken) and lets you pass a model parameter to track costs per AI model:

?url=https://example.com&model=claude-sonnet-4

Your dashboard shows per-model cost breakdowns, so you know exactly what each URL costs across different models. This is useful when you're comparing models or optimizing your agent's budget.

Truncation for Predictable Costs

Need to cap token usage? Use max_tokens:

?url=https://example.com&max_tokens=2000

StripFeed truncates at paragraph boundaries, so you never get a cut-off sentence. The response tells you if content was truncated, so your agent can decide whether to fetch more.

The Bottom Line

If your AI agent reads web pages and you're feeding it raw HTML, you're paying 3-5x more than you need to. The fix is straightforward: convert to Markdown before sending to your model.

Sign up for free and start saving tokens today. The free plan includes 200 requests/month, enough to test the impact on your pipeline.