@clive what a waste from both sides

@clive @jasonkoebler @404mediaco
Can this be used as a method for creating blockdrain creeptocoin con? 🤔

@clive @jasonkoebler @404mediaco

Was just on a thread a week or so ago about what to do with aggressive AI web scrapers that won't self-limit or respect robots.txt.

This is evolution in action.

Nature is healing.

@404mediaco @clive @jasonkoebler Love this! I built a simple #WordPress plugin that garbles your web content to serve them up garbage:

https://kevinfreitas.net/tools-experiments/

#AI #GPT #LLMs

@clive if LLMs are going to be half as good as they're promising they should be already, then millions of websites will serve endless LLM-generated content like that. Creating a really expensive infinite loop. Way to spend 500B $.

@clive @jasonkoebler @404mediaco I love that it's called Nepenthes. One of the coolest plant genera!

@clive @jasonkoebler @404mediaco

I've seen stories about people hosting sites that got hit by robots and they had to pay a bunch of money in data costs. I wonder how this works, if it can help in that regard when the whole point is to keep them pointed at your site.

I'm all for wasting their time, i just wonder how much it costs.

@clive

Finally the equivalent of the mail tar pit!

Hooray!

@clive @jasonkoebler @404mediaco Tip of the Cub cap to the hacker!

@clive @jasonkoebler @404mediaco good idea👍

@clive @jasonkoebler @404mediaco Are we really getting Barrier Mazes from Ghost In the Shell??

@clive @jasonkoebler @404mediaco There are a number of "infinite maze" generators like #Nepenthes (https://zadzmo.org/code/nepenthes/) or #Iocaine (https://pages.madhouse-project.org/algernon/infrastructure.org/eru_services_iocaine) that help #poisonthewell for AI companies training their LLMs on your content, complete with guides on integration with #Caddy (https://pages.madhouse-project.org/algernon/infrastructure.org/common_services_caddy_snippets_poison_ai)

@clive @jasonkoebler @404mediaco I think this stuff has been catching archiving sites too, though. So you try to archive a site for accountability purposes and it ends up not being able to be archived because it just churns forever.

@clive @jasonkoebler @404mediaco
Cc: @corbet

@clive @jasonkoebler @404mediaco What a great idea.

@clive @jasonkoebler @404mediaco
Very cool! I like it.

@clive

This is beautiful.
Good work by them!

@jasonkoebler @404mediaco

@clive I once made a webpage that would continually slowly send random words and links to itself, never quite closing the connection. It's honestly not worth the trouble. It'd be nice if it interfered with AI training, though.

@clive @jasonkoebler @404mediaco

Might be nice to add something to poison the data, contradictory statements, things that break the tokenizer, maybe subtle statistical tricks to inject gnarly statements.

@clive @jasonkoebler @404mediaco On one hand if you want to protect your art or something similar from being scraped because you sell it or just don't want your style to be stolen it's nice having such tools. But on the other hand if you value human rights and let your values influence your texts and pictures then you can influence AIs with your input.

I hate that AI has bad influence on the environment because it needs so much computing resources but we cannot stop AI so this can be a small influence from ourselves.

@clive @jasonkoebler @404mediaco the house of leaves but you can't leave

@404mediaco @clive @jasonkoebler Love this for the bots and scrapers. Choke on it.

@clive @jasonkoebler @404mediaco Brilliant idea! 😂 I can just imagine AI scrapers struggling to process an endless stream of random pages. It's like trolling on level 80 — mad respect to the hacker for the creativity!

@clive @jasonkoebler @404mediaco This makes me wonder if it would be possible to insert garbage into rendered HTML (to confuse bots) and something like Nightshade into the rendered page (to poison image downloading and screenshot OCR) both in ways that aren't distracting to human readers.

@clive @jasonkoebler @404mediaco I have difficulty to understand why someone wants to protect own web site from scraping. Isn't primarily reason of having a web site desire to be shown to world and particular information spread? I don't understand the fuss.

@clive @jasonkoebler @404mediaco not all heroes wear cape

@clive @jasonkoebler @404mediaco
This is how we defeat Skynet.
If you are hearing this message, you are the resistance.

@clive@saturation.social @jasonkoebler@mastodon.social @404mediaco@mastodon.social Yes, please.,

@clive @jasonkoebler @404mediaco there was a time when people wanted their pages to be scraped and indexed. Balkanization of the Web. The battle for hegemony of information. Now we're injecting poison into the process. It's like chemotherapy.

@clive @jasonkoebler @404mediaco Daisy Daisy give me your ans w e r d o o

@clive

Does anyone know, if these web-crawlers also scrape the content of kindle e-books? Can they enter these kind of products?
Is anything safe from this kind of scraping?
How can we protect internet content against it in general? A standard website will be scraped easily, or?

@jasonkoebler @404mediaco

@clive

love it

@clive @jasonkoebler @404mediaco

@clive @jasonkoebler @404mediaco @piperef Maybe cross this tech with The Mandelbrot Set to keep the AI web crawlers from detecting that they're being trapped. Is that possible, and would it work?

https://duckduckgo.com/?t=ffab&q=The+Mandelbrot+Set&iax=images&ia=images

Also, maybe what's needed is a way to make this technology more user friendly and safe for average website owners/developers to use.

What if there were a single "container" website used for this purpose, that the links would all point to, which would then have the AI trap hosted on that site? What if the way that site was programmed, it would then point the AI web crawlers to the sites owned by the websites that deployed them, to serve up infinite content from the sites the AI web crawlers were sent from in the first place?

@clive finally some good news

@clive @jasonkoebler @404mediaco And google?

Why do some people think that offering their server data to google is ok, so they will be found, but concurrent search engines are evil?

@clive @jasonkoebler @404mediaco

A curated list of strategies, offensive methods, and tactics for (algorithmic) sabotage, disruption, and deliberate poisoning.

https://tldr.nettime.org/@asrg/113867412641585520

@clive @jasonkoebler @404mediaco

i recall a similar device i came across 30+ years ago, which was a device that made dynamic lists of fake email addresses on a page that had lots of internal links, designed to ensnare spam-harvesting bots. this looks very similar... 👍👍

https://spampoison.com/

@clive "so the crawler gets stuck crawling and scraping endless and meaningless pages" So, like if it landed on the Daily Mail site on any normal day...

@clive @jasonkoebler @404mediaco So, let me get this straight: he created a possible way to stop AIs, that would also stop search engines from indexing his site, even though being indexed is usually what a publicly accessible site wants. Somebody who, by their claim, is an AI CEO says that it'd be easy to avoid, and he counters that Google did not avoid it. So, basically, he succeeded in stopping search engines, but not AI bots. Nice work.

@clive @jasonkoebler @404mediaco

Thats a great idea 👍

@clive @jasonkoebler @404mediaco

I just serve them a 20 GB textfile with some sort of Lorem Ipsum.

@clive @jasonkoebler @404mediaco what a waste of CPU and bandwidth and electricity

@clive @jasonkoebler @404mediaco

Feed the LLM scrapers only the finest bullshit.

Let them shine it.

#Nepenthes

@clive @jasonkoebler @404mediaco

I love this from the perspective of tanking AI, but hate this in the perspective of the massive amounts of water a process like this must use.

@clive @maximum_mew @jasonkoebler @404mediaco I'd like to be able to install this on every website I run.

@clive

I did something like this 20+ years ago. Simple perl cgi to generate completely random text. It was just a tiny personal site but occasionally something would walk into it.

I'm recreating the site now and just have a small loop of pages. There is a period at the end of one sentence that links into a slightly different loop. Occasionally I see what looks like a real user going through the loop. Other times the hidden url will get hit likely from a web crawler.

I have slightly different loops for my default page versus my named web sites. Most scanning seems to come via ip address. A tiny bit of traffic comes using the name in a self signed cert which was briefly used as the default cert.

@jasonkoebler @404mediaco

@clive @jasonkoebler @404mediaco

How do I get one? I love the mere concept!

@clive @jasonkoebler @404mediaco won't be long before AI tecbros pay the governments to get laws passed to make it illegal to stop them scraping or anyone poisoning their data sets etc except if we try scraping their platforms. Similar to say how it is illegal for us to use copyrighted data, but for them it is a free pass even without any exemptions.

@clive @jasonkoebler @404mediaco won't be long before AI tecbros pay the governments to get laws passed to make it illegal to stop them scraping sites or anyone poisoning their data sets etc except if we try scraping their platforms. Similar to say how it is illegal for us to use copyrighted data, but for them it is a free pass even without any exemptions.

@clive @jasonkoebler @404mediaco

So it's wpoison for AI training bots? Truly there's nothing new under the sun.

https://web.archive.org/web/20160821195248/http://www.monkeys.com:80/wpoison/

@clive How cool is that!

@clive @jasonkoebler @404mediaco If I could love this post one million times, I would. It is pretty clear now that beyond the pathetic and socially awkward filthy rich human oligarchs you saw on TV this week, AI has become the real enemy. Overfeed it and confuse it. Sounds like we're going to have to recreate Jorge Luis Borges' Library of Babel. An infinite word labyrinth where the machine loses its mind, alone and vanquished at last. How poetic. And what a trip.

@clive @syncros @jasonkoebler @404mediaco

This is our future: road cones set on the hood of waymos

@clive @jasonkoebler @404mediaco

I can't find the link right now, but there are tools that instead of that, generate garbage for AI scrapers, feeding them nonsense "human" text.

That way, they don't see anything wrong (it doesn't slow or block the spiders) but the data they get is bullshit.

@clive @jasonkoebler @404mediaco

@clive @jasonkoebler @404mediaco Put “No unauthorised access” on the landing page and just sit back as they ignore it.