🦞 x-tweet-fetcher

Fetch tweets, lists, articles, and WeChat content — with smart backend routing.

Three backends · Auto fallback · Works everywhere (VPS / Mac / Windows / CI / Claude Code / OpenClaw)

Quick Start · Backends · Capabilities · Self-hosted Nitter · Claude Code & CC

---

😤 Problem

You: fetch that tweet / list / article for me
AI:  I can't access X/Twitter. Please copy-paste the content manually.

You: ...seriously?

X has no free API. Scraping gets you blocked. Browser automation is fragile and won't work in headless environments.

x-tweet-fetcher solves this with smart backend routing: Nitter for zero-dependency speed, Playwright for full-feature coverage, auto fallback between them.

🔀 Three Backends

# Auto mode (default) — Nitter first, browser fallback
python3 scripts/fetch_tweet.py --user elonmusk

# Nitter only — zero dependency, no browser
python3 scripts/fetch_tweet.py --user elonmusk --backend nitter

# Browser only — full features (lists, articles)
python3 scripts/fetch_tweet.py --list 1455045069516357634 --backend browser

Backend	Deps	Speed	Features
nitter	None (stdlib only)	⚡ Fast	Timeline, search, replies, profile, mentions
browser	Playwright/Chromium	🐢 Slower	Everything above + Lists + Articles + fetch_china
auto (default)	Best available	⚡→🐢	Tries nitter first, falls back to browser

OpenClaw users: Playwright + Chromium are built-in. --backend auto just works — no extra install needed.

📊 Capabilities

Feature	Backend	Output
Single tweet	FxTwitter (always)	text, stats, media, quotes
Reply comments	nitter / browser	threaded comment list
User timeline	nitter / browser	paginated tweet list
@mentions monitor	nitter / browser	incremental new mentions
Keyword search	nitter / browser	real-time tweet stream
X Lists	browser only	list member tweets
X Articles	browser only	full long-form content
User profile analysis	nitter + LLM	MBTI, Big Five, topic graph
WeChat article search	Sogou (direct HTTP)	title, url, author, date
WeChat/Weibo/Bilibili	browser only	via fetch_china.py
Tweet growth tracker	FxTwitter API	growth curves, burst detection

For AI Agents: All output is structured JSON. Import as Python modules for direct integration. Exit codes are cron-friendly (0=nothing new, 1=new content).

🚀 Quick Start

Single tweet (zero setup)

# Works immediately — no Nitter, no browser needed
python3 scripts/fetch_tweet.py --url https://x.com/elonmusk/status/123456789

Timeline, search, replies

# Set your Nitter instance URL (for nitter/auto mode)
export NITTER_URL=http://127.0.0.1:8788

# User timeline
python3 scripts/fetch_tweet.py --user elonmusk --limit 20

# Keyword search — real-time tweets
python3 scripts/nitter_client.py --search "AI agent"

# Tweet replies
python3 scripts/fetch_tweet.py --url https://x.com/elonmusk/status/123456789 --replies

# @mentions monitoring (cron-friendly)
python3 scripts/fetch_tweet.py --monitor @yourusername

# User profile analysis
python3 scripts/x-profile-analyzer.py --user elonmusk --count 100

Lists & Articles (browser backend)

# X List — requires Playwright
python3 scripts/fetch_tweet.py --list 1455045069516357634 --backend browser

# X Article
python3 scripts/fetch_tweet.py --article https://x.com/user/article/123 --backend browser

# WeChat / Weibo / Bilibili
python3 scripts/fetch_china.py --url "https://mp.weixin.qq.com/s/..."

WeChat search (always zero-dep)

python3 scripts/sogou_wechat.py --keyword "AI Agent" --limit 5 --json

🖥️ Works with Claude Code / CC

Since x-tweet-fetcher has zero mandatory dependencies, it works perfectly in constrained environments:

Environment	nitter mode	browser mode	Notes
Claude Code (CC)	✅	❌	No browser runtime
OpenClaw	✅	✅	Playwright built-in
VPS (headless Linux)	✅	✅*	*needs pip install playwright
Mac / Windows	✅	✅*	*needs pip install playwright
CI/CD pipelines	✅	⚠️	Possible but heavy
Docker containers	✅	⚠️	Needs Chromium in image
Termux (Android)	✅	❌	No Chromium

# In Claude Code (nitter mode, zero deps):
export NITTER_URL=http://your-vps:8788
python3 scripts/fetch_tweet.py --user YuLin807 --limit 10

# In OpenClaw (auto mode, full features):
python3 scripts/fetch_tweet.py --user YuLin807 --limit 10
# → auto-detects Nitter, falls back to Playwright if needed

🔧 Self-hosted Nitter Setup

⚠️ Public Nitter instances are dead or unreliable (as of March 2026). Self-hosting is the only reliable option.

Why you need this

Twitter removed guest API access in 2023. Public Nitter instances get rate-limited because thousands of users share a few accounts. Your own instance = your own rate limits.

5-minute setup guide

1. Install dependencies

# Ubuntu/Debian
sudo apt install -y redis-server libpcre3-dev libsass-dev

# Install Nim
curl https://nim-lang.org/choosenim/init.sh -sSf | sh
export PATH=$HOME/.nimble/bin:$PATH

2. Build Nitter

git clone https://github.com/zedeus/nitter
cd nitter
nimble build -d:release
nimble scss
cp nitter.example.conf nitter.conf

3. Get X session cookies

Use a secondary account (not your main).

1. Log into X in browser → DevTools → Application → Cookies → x.com
2. Copy auth_token and ct0
3. Create sessions.jsonl:

{"name":"myaccount","auth_token":"YOUR_AUTH_TOKEN","ct0":"YOUR_CT0"}

4. Configure

[Server]
address = "127.0.0.1"  # Local only!
port = 8788

[Config]
hmacKey = "$(openssl rand -hex 32)"

[Tokens]
tokenFile = "sessions.jsonl"

5. Run & test

sudo systemctl start redis-server
./nitter

# Test
curl http://127.0.0.1:8788/YuLin807
export NITTER_URL=http://127.0.0.1:8788
python3 scripts/nitter_client.py --search "test"

Security

• Bind to 127.0.0.1 only — never expose to public internet
• Use a secondary X account — session token gives full access
• Session tokens last ~1 year

📐 How It Works

                    ┌─────────────┐
 --url              │  FxTwitter  │  ← Public API, no auth needed
                    │  (free)     │
                    └──────┬──────┘
                           │ JSON
              ┌────────────┴────────────┐
              │    --backend auto       │
              │  ┌───────┐  ┌────────┐  │       ┌──────────┐
 --user       │  │Nitter │→→│Browser │  │       │  Agent   │
 --replies    │  │(fast) │  │(full)  │  │──────▶│  (JSON)  │
 --monitor    │  │ 0 dep │  │Playwrt │  │       │          │
 --search     │  └───────┘  └────────┘  │       └──────────┘
 --list       └─────────────────────────┘
 --article
              ┌─────────────┐
 sogou_wechat │   Sogou     │  ← Direct HTTP, no API key
 fetch_china  │  (search)   │
              └─────────────┘

• Single tweets: FxTwitter — always works, zero auth
• Timeline / Replies / Search / Mentions: Self-hosted Nitter or Playwright browser
• Lists / Articles: Playwright browser (Nitter doesn't support these)
• WeChat / China platforms: Sogou search + fetch_china.py

📦 Requirements

Python 3.7+     (that's it for nitter mode)

Mode	Extra requirement
--backend nitter	Nothing (Python stdlib only)
--backend browser	pip install playwright + playwright install chromium
--backend auto	Uses whatever is available

⏰ Cron Integration

Exit codes for automation: 0=nothing new, 1=new content, 2=error.

# Check mentions every 30 min
*/30 * * * * NITTER_URL=http://127.0.0.1:8788 python3 fetch_tweet.py --monitor @username

# Discover tweets daily
0 9 * * * python3 nitter_client.py --search "AI Agent" >> ~/discoveries.jsonl

🤝 Contributing

Issues and PRs welcome! Core platforms:

• X/Twitter — Nitter + Playwright backends
• WeChat articles — Sogou search

Other platforms welcome as community PRs.

🙏 Acknowledgments

• Nitter by zedeus (12.6k ⭐) — self-hosted Twitter frontend
• FxTwitter — public API for single tweet data
• Playwright — browser automation for full-feature coverage
• OpenClaw — AI agent framework

📄 License

MIT

---

Three backends. Auto fallback. Works everywhere. 🦞

GitHub · Issues · OpenClaw Q&A