Cloudflare Guest Protection

Due to the heavy rise in AI scraping and bot scraping in general, I have activated a rule that should only affect guests, not logged in members, where every guest session will go through the Cloudflare managed challenge process. Known good bots are bypassed, so it doesn't affect legitimate search engine bots, just all the other nonsense.

To put it in perspective, daily uniques went from 300k to 1M. We should have, normal real traffic, 200k - 300k daily. The server will take it all, no issue with that, but its still just abuse right now and its a global problem.

If this doesn't work as intended, let me know, but if logged in you shouldn't have any cloudflare automated challenges popping up. If you have to login, then you will likely get it before you're able to login.

Hello to the tens of thousands of lurkers out there!

You can see a huge difference in the members and guests online now, legit bots, guests and members, and all the crazy stuff going in circles.

I am testing an alternative... basically testing most restrictive to least restrictive, yet does the job I need it to do.

Right now, I have a challenge in place on the search function for guests, which is where most of the abuse is stemming from. Basically, bots using search with a username to grab all that content directly, rather than scraping posts and then pulling data from comments. To me, this is a recent AI issue that came to light, where AI's are putting together what is, anonymous content posted online, but from that content per username, they are matching it up with real people by building complex profiles and I guess, using a complex algorithm to determine probability.

It was raised as an issue recently, and I am seeing a specific pattern being used recently which would achieve that aim, if looked at how I am seeing the logs. So I have stopped that happening in this current test, trying to not limit good bots from doing good things, but stopping bad bot behaviour.

anthony said:

Basically, bots using search with a username to grab all that content directly, rather than scraping posts and then pulling data from comments. To me, this is a recent AI issue that came to light, where AI's are putting together what is, anonymous content posted online, but from that content per username, they are matching it up with real people by building complex profiles and I guess, using a complex algorithm to determine probability.

Click to expand...

Can I ask, just out of interest, why they are doing this? What do they hope to achieve with that data?
Are they trying to work out who the users are by what we've said where we live, what work we do, etc?

There is talk about companies and Government using AI to piece together all online information, trying to find patterns to assign anonymous data to known data, thus a known person. Basically, anonymity online will truly be fictitious. There are ways to beat it on social media and communities, such as implementing an AI between what the user writes and what is posted, but then that has issues too.