the fix
DuckDuckGo returns 202 to your scraper — what to use instead
html.duckduckgo.com returns a 202 bot-challenge page to scripted GET requests — not an error you can retry past, just a page with no results in it. Google and Bing block curl outright. Every free HTML search endpoint is a cat-and-mouse game with a team whose job is to stop you.
What people try first
| approach | what happens |
|---|---|
| html.duckduckgo.com | 202 challenge page. No results. |
| Google or Bing HTML | Blocked, and scraping them breaks their terms. |
| SearxNG instance | Workable if you host it, but public instances rate-limit hard. |
| A search API | Clean JSON, no challenge pages, no proxy budget. |
The one-call fix
One request, unified JSON, no proxies or parsers to keep alive. 2 credits a call once you are on credits, and free for your first 5.
curl -H "x-api-key: $SS_KEY" \ "https://api.socialsearchapi.com/v1/web/search?query=llm+agents"
Signup is an email and a 6-digit code — no card — and gives you 5 free calls on any platform, so the command above runs as-is.
Real response from that endpoint, captured 2026-07-25:
{
"success": true,
"platform": "web",
"data": {
"total": 3,
"items": [
{
"platform": "web",
"type": "result",
"title": "Reddit API Alternatives 2026 — Pricing & Limits Compared",
"text": "Estimate your monthly Reddit API spend in seconds. Start fetching Reddit data — $0.10 in free credits. Six ways to fe...",
"url": "https://www.redditapis.com/reddit-api-alternatives",
"created_at": null,
"extra": {
"score": 0.9053309
}
}
]
},
"credits_used": 2
}Related
Search Web / News — full guideAPI referencePricingReddit returns 403 to curl — why, and how to fix itHow to search Reddit without logging inInstagram API alternative for reading public postsCheap X/Twitter search API alternativesTikTok scraper API for videos and hashtag search
Last verified 2026-07-25. Platform terms and pricing change often — if something here has gone stale, tell us at [email protected].