@matasoft @clive @jasonkoebler @404mediaco If all LLM platforms always provide direct links it would indeed bring people to your site. But the fact is that most LLM's just steal your content without giving any credit. That's what this is for.

@matasoft @clive @jasonkoebler @404mediaco there are masses of AI crawler bots that can easily overwhelm a web site with traffic. They don’t throttle or follow limits or respect robots.txt or other conventions. They’ll easily overwhelm available bandwidth and take down a small web site.

It’s 100% reasonable (and necessary) to consider them hostile.

@matasoft disrespectful scraping for LLM removes attribution, is non-consentual, has no methods for rectification (misinfo), mixes your data with others' (sometime criminally sourced) and often is implemented so badly that it causes load/stability/cost issues to the sites. And this is not "one shot", but happens all the time. No one wants this except the grifters.

@matasoft
Guess people want the world to know it's their work. Also there was a post from an admin here complaining that AI scrapers aren't as smart as normal e.g. search engine scrapers and come back every 10 minutes to scrape the exact same data, costing his small instance ridiculous money. And stuff like this here, no I don't need my random musings distributed elsewhere.

Assuming you were asking in good faith.

@clive @jasonkoebler @404mediaco