Experiment insights

The bot invasion continues, but my newest experiments in combat tactics are at least revealing some information. The Cloudfare service I'm trying out is a mixed bag, to be sure, and I'm still not sold on it being all that useful, but it is able to see through some of the bot camouflage. That in and of itself might be helpful in creating IP blocks that actually work. We'll see.

For example, there's a hit in my access logs from about an hour ago ostensibly from Senegal on a certain IP, using Chrome version 103 on Windows 10. It's obviously a bot because it matches certain patterns in my access logs, but it successfully oozed through all the "bots keep out" forcefields and convinced servers it was a human user. The extra layer of filtering reveals that this bot was actually in Venezuela on a different IP, using Chrome 110 (headless, which means no user interface, which means entirely under-the-hood snooping), using or simulating Windows 10. Another hit claimed to be from Mexico on a certain IP and was really from El Salvador on another, pretending to be on a MacOS.

It does appear to be intercepting a lot of bot traffic, but I'm still a bit unclear on what's being reported; a lot of these, I think, are bots not trying to disguise themselves (like Googlebot) and thus hit my robots.txt file and see the "keep out" sign and obey it. These would be stopped anyhow, I'm just seeing more data on it this way. It's also reporting more bandwidth usage than my actual usage seems to be, which hopefully indicates intercepted bots that don't make it through the filter? It's going to take a few more days worth of data to give me any sense of difference its making in terms of stopping bots in their tracks.

Meanwhile, I am also considering turning this site into a "members only" sort of thing that requires a login for anyone to see it. I don't like the idea conceptually, but password-blocking is a sure way to eliminate bot theft. At the moment, I can only assign members manually, from my admin interface, so that's a non-starter. I would need to code a new feature into the site guts to allow people to sign up from outside, and then add stuff to prevent bots from signing up (or at least minimize them). I may have a low-tech workaround for that, based on the idea that bots, especially these newfangled scraper bots, will look for more technical means of defense to thwart and might miss a simpler approach.

How would people feel about being forced to log in? I well understand that the average websurfer has the attention span of a gnat, and that any obstacle might be enough to cause them to move along to something else, but this seems like a simple enough ask. Any thoughts?

I'll try and implement that simple workaround login test tomorrow/later in the week. Can't start on it now or I'll end up losing track of time and I have to be out of here in another hour or so because I start my 2026 umpiring this evening. It'll be cold, the field will be wet, and the teams playing don't include any of my favorites (though no serial troublemakers, either). I'm not exactly excited about it, but it will be nice to be back on the field again.