Crawler · public policy
Our crawler policy
WaveScorePro publishes only directly measured digital audience numbers. To do that we probe public Icecast / Shoutcast stats endpoints on the open web. This page documents exactly what we do — so any operator worried about the bot in their access log can verify our behaviour and contact us if they'd like to be excluded.
How to identify us
User-AgentWaveScorePro-Crawler/1.0 (+https://www.wavescorepro.com/crawler)
Contactcrawler@wavescorepro.com
robots.txtHonoured — cached per host
Rate limit1 request / 30s per host
Backoff on 429/5xx60s → 3600s (exponential)
Endpoints we probe
We only read the endpoints your streaming server publishes for anyone. We do not attempt authentication, do not scrape admin panels, do not download audio for storage.
Icecast /status-json.xslShoutcast v1 /7.htmlShoutcast v2 /stats?sid=1&json=1ICY in-band metadata (Icy-MetaData: 1)
Anti-gaming sanity layer
Every crawled number passes through three defences before it appears on any public surface:
- Endpoint authenticity — the response must parse as a structurally valid Icecast/Shoutcast payload.
- Endpoint dedupe — if two stations point at the same mount-point, only the first-verified one counts. Cloning a station page cannot inflate two rows off one stream.
- Spike freeze — a listener count more than 3× the trailing 24h mean is frozen to the 90-day P95 and flagged for review.
Coverage right now
Stations known65
Currently measured (DIRECT)3
Coverage ratio5%
Hosts tracked (crawl-loop)16
Hosts in backoff0
How to exclude us
Add this to your robots.txt:
User-agent: WaveScorePro-Crawler Disallow: /
Or email crawler@wavescorepro.com and we'll add a manual block within 24h.