Tag: Site issues
Out of drydock
Well, that was fun. In a tear-one's-hair-out kind of way.
As you may have noticed, the site was down for about a full day. I broke it by fixing it. The nature of more complicated PHP/MySql platforms running on infrastructure that never stays the same means that when you want to add a new feature or adjust an old one, you run the risk of "version conflict." The issue that begat this latest round of repair was a mysterious database error that I could find no cause for. I decided to update the database to run on newer PHP. Probably fixed the error issue, but since it broke more of the site I couldn't tell for sure.
Anyway, in the end I upgraded everything, relaunched this ship from scratch, and if I've done it right, you all won't even notice anything is different. (Except anyone who might have signed up for updates through Feedrabbit. You all got spammed with the whole history of posts. Sorry about that.)
I know, though. Because I have less hair now.
No Comments yetThe more you overtake the plumbing...
While the bot war has entered a lull—we'll need to wait until the calendar flips in order to see a good data benchmark to confirm that the mitigation efforts are consistently working—I've been spending some time addressing some collateral damage that's piled up since major war efforts began a few weeks back.
This site is built on a pretty great platform, but Internet infrastructure technology keeps changing and thus when I tweak one thing to address issue A, a second thing breaks causing issue B; in fixing issue B, I discover that what used to work as a fix doesn't anymore because PHP syntax has changed and now we need an update to the platform guts to satisfy that. But then we get a new conflict with issue C, because I built that section years ago when things were still running on PHP 5 and the upgrade is confused now. It's a little like the nursery rhyme about the spider and the fly.
Anyway, most of these headaches won't be evident to you the reader (I hope) unless you happen upon the site right at the moment I've broken something; most of this is under-the-hood stuff. But having solved most of the problems, one remains and it's annoying the heck out of me because I can't figure out what's broken. The back end keeps throwing database errors at me even though function doesn't appear to be at all impaired and everything renders fine.
I could just choose to live with it, I guess, but you know me—am I going to do that? No, of course not. Perfectionist brain won't allow it.
All of which is to say that:
- We may experience some downtime in the next couple of days while I attempt a rather extreme cleaning out of site guts in search of the gremlin.
- It may turn out that I'm better off upgrading the platform, which would be good but that always generates more unintended consequences.
- Meanwhile, in clearing up collateral damage I reconfigured my RSS backend so that it will work with my old email notification system despite Cloudfare stuff, so I'm junking that MailerLite daily email thing and you all should go back to getting single-post updates as they go live. (MailerLite was more trouble than it was worth, it's not intended for little stuff like this.) For the moment, there's no "sign up for email updates" button that works (the existing one still goes to MailerLite and will do nothing EDIT: The button now goes to Feedrabbit again as a functional stopgap), so if you want to add your name to the list, say so in the comments or email me a note.
Now, back to gremlin hunting...
No Comments yetMélange of miscellany
Kudos to the great people of Minnesota, who go out in the cold to document the fascism
Rather than pick a topic and dive in I'm going with a potpourri of assorted notes on various things today...
Dateline: Occupied Minnesota
-
The campaign of state-sponsored terror continues in occupied Minneapolis, and thankfully the good people who live there are out documenting it. One video made it to the cablewaves of the Chicago-based allegedly-centrist-but-Republican-slanted station NewsNation, which showed it over an interview with Congresswoman Mary Miller (R–IL). Rep. Miller said the woman shown in the video being abducted out of her car while trying to drive to a medical appointment deserved to be manhandled and abused because "she's here illegally [and] probably getting free health care." She later admitted that she doesn't know who the woman is and thus has no earthly idea if she's here illegally or not, but she also says, "who cares? She's breaking the law and resisting arrest." For the record, the woman was later identified as a biracial U.S.-born software engineer and ICE had to release her.
The video, which is all Rep. Miller had to go on, shows no lawbreaking whatsoever on the woman's part; it shows ICE acting illegally, though, breaking her car windows and abducting her rather than allowing her to move along her way. Was she being arrested for cause? Was she to be charged with something? Would the charge be, say, asking an ICE patrol why she was being hindered from traveling to her appointment? That's the "lawbreaking" Rep. Miller sees and says "who cares?" about? This interview is the first and thus far only time I've ever seen Rep. Mary Miller, I'd never heard of her before, but it's plenty sufficient to reveal her as a racist, authoritarian abettor of criminals with no respect for law or her oath to protect and defend the Constitution of the United States. In a sane world, she would be censured, removed from committees, or even expelled from the House by her fellow Congresspeople for what she said in this interview.
Miller represents a gerrymandered district created in the 2020 redistricting that moved Illinois 15 from the southeastern corner of the state to a rural expanse in the center that is nearly bisected by population centers. If you live in Decatur, Springfield, or Champaign, congratulations, you are not Miller's constituent, though you are completely surrounded by those who are. If you live in East St. Louis, Bloomington, or Peoria, you're less than 20 miles from IL–15. The 2020 census took a seat away from Illinois, necessitating a redistricting, and the state redrew its map to group cities together as much as possible in a "fight back" gerrymander. It gave the state three more Democratic seats in the House but eliminated toss-up districts and made the three remaining R districts deeply, deeply Republican. This new blood-red district had two incumbents, one holding the seat lost in the census, and its voters reelected Miller over the more moderate R and then overwhelmingly reelected her again. Congrats, IL–15, you people are batshit crazy. At best.
- In another incident on Monday, ICE agents kidnapped a black woman, a U.S. citizen, in occupied St. Paul. It wasn't clear if the damage to her car seen in the video was caused by ICE or not, but rather than assist someone after an automotive collision of some kind the agents abducted her, threw her into an unmarked vehicle, and drove off.
- Also Monday, ICE agents abducted two teenage employees of a Twin Cities area Target store, tackling one of them to the ground and beating him, only to dump them out of the unmarked vehicle, bloody and sobbing, eight miles away when they were satisfied that the teens were American citizens. (Video is on X, so I'm not linking it. Screw you, Elon.)
- Three Minnesota school districts (and counting?) are now accommodating remote classes as it is unsafe for students to attend school. ICE has abducted parents, tear-gassed playgrounds, and generally terrorized various Minnesota schools this month in their alleged quest to deport immigrants.
- A Minneapolis resident, abducted by ICE for following an ICE vehicle and alerting neighbors to ICE's presence—agents stopped her car, broke the driver and passenger side windows, and forced the two occupants from the car—said that while being forcibly taken in an unmarked vehicle to a nearby Federal building agents told her, "you guys have to stop obstructing us, that's why that lesbian bitch is dead." The threat, the misogyny, the bigotry, the callous disregard for law, the small-minded insecurity, all there in one quote from a government-sanctioned thug during an illegal arrest. Before being released, agents apparently offered at least one of the two abductees money if they would name or identify other protesters.
- Congresswoman Robin Kelly (D–IL 2, south Chicago suburbs) has spearheaded a move to impeach Kristi Noem over her use of ICE thugs in Minnesota and elsewhere. More power to you and your colleagues, Rep. Kelly. It won't succeed in this House, but I applaud the effort and want to see more of this. Just because Speaker Johnson won't allow such things to be voted on doesn't mean we shouldn't be doing them every damn day.
- An activist in the Netherlands was given a list of more than 4,000 names of people working as ICE agents or support personnel. He put it online. Since these people have no business trying to hide their identities in the first place, I'll link to it. The site is slow to load, I imagine it's getting a bit of strain put on its server.
- I look forward to the massive number of lawsuits that will eventually be filed against DHS, the least of which will be a plethora of demands for financial restitution for property damage to the various cars agents have rammed, broken windows of, sliced seatbelts in, and, you know, shot.
The hot stove league
-
With the signing of free-agent third baseman Alex Bregman, the Chicago Cubs have bumped incumbent third-sacker Matt Shaw to the bench. Shaw is a MAGA ideologue who left the team to attend the funeral of Charlie Kirk and again during the pennant race to go to a MAGA rally in Arizona. Thus, when the Bregman signing became official we got this outstanding post on BlueSky:

Our buddy Craig Calcaterra followed up on that with this sentiment:
I suppose Shaw will now be a super utilityman. Which makes me REALLY want the Cubs to acquire a better utilityman such as Santiago Espinal or someone like him so Shaw can be replaced, in the same offseason, by both a Jew and a Latino guy. That'd probably break his fascist ass.
Schadenfreude for the win.
-
The Kansas City Royals are the latest team to do something stupid with their field dimensions. That's my bias, of course, that it's stupid. The Royals are moving the fences in at Kaufmann Stadium, shortening the alleys between the foul poles and dead-center field by ten feet. Not satisfied with that, they are also making the fence height 18 inches shorter. KC's general manager, J.J. Picollo, claimed he wasn't "trying to jump-start our offense," which is silly, of course he is. But the thing is, Kaufmann has always been a great hitters' park. It just hasn't been a great home run park. Especially in the days when it had AstroTurf, but even with grass, KC's was a terrific park to hit doubles and triples in. A big outfield means potentially fewer homers, but it also means more base hits—outfielders have more ground to cover, balls are going to fall short of their positioning or go over their heads more often than they would in smaller outfields. Also, a curved symmetrical outfield wall meant any roller that hit the wall had the opportunity to hug the wall as it rolled on rather than carom back to an outfielder. I haven't seen whether or not they're trying to keep the curvature of the wall, but no matter what it won't be as prevalent since the degree of curve will be lessened. This is a move intended to make home runs easier to hit and to thus encourage batters—Royals and opponents—to keep up the dumbness that has made for less interesting baseball since the post-strike 1990s. That is, the all or nothing, "three true outcome" style offense that has reduced balls in play, skyrocketed strikeouts, and massively devalued defensive skill, particularly for outfielders.
I've said it before, and I'll say it again: Home runs are boring. Compared to most other ways to score, unless it's a walkoff ending a game, a homer is dull. It's a flash-in-the-pan event, a potential rally-killer, while a string of base hits keeps pressure on the pitcher and the defense. Other than a bases-loaded walk/hit batter, a balk, or, god forbid, a pitch timer violation, the home run is the least interesting way to score. Teams ought to be doing what the Orioles did a few years back and making their outfields bigger. Encourage more contact, encourage smart baserunning, make outfield defense important, and above all, make the game less reliant on brute force. Rob Deer was an interesting player because there just weren't very many Rob Deers. Nowadays every team has at least two of him. The world champion Los Angeles Dodgers had five players top 120 strikeouts in 2025. Five! (Your Seattle Mariners only had one, which is a big reason they were so much better in ’25 than in prior seasons.)
I realize I'm never likely to see a team like my beloved 1985 Cardinals ever again, but can we stop trying to make baseball dumber? Please?
- The Washington Nationals are the latest team to ditch their cable television contract, leaving the Mid Atlantic Sports Network and turning over their TV rights to Major League Baseball. The cable TV model is quickly dying and I am here for it. What remains to be seen is how MLB is going to be handling the various teams (now including Your Seattle Mariners) they need to televise. Presumably they will find a cable outlet in each of the markets they can pipe feeds to, but really the need is in streaming. Because Commissioner Dumbass shot himself in the foot trying to extort a better playoff TV deal from ESPN, he ended up losing revenue and to try and make up for the loss in the short term sold ESPN the streaming rights for what had been MLB.TV. Until very recently, MLB.TV was only meant for subscribers to watch out-of-market teams and that's what ESPN now has control over; whether MLB will retain these individual teams' in-market streaming rights or lump them in with the ESPN deal is unclear. We'll find out in a couple of months. Regardless, the death of the cable model means an end to the stupidity of making it difficult/impossible for local fans to watch their own team without paying through the nose for a cable/satellite package. That stupidity remains for playoff games, though, so there's still a ways to go. But it's more evidence that Commander Data was right in Star Trek: TNG when he mentioned that broadcast television didn't last in any significant form beyond the year 2040.
Site tweaks and email issues
- I have succeeded (I think) in eliminating the duplicate email problem with the new daily update email subscription thing. However, I have in the process discovered that the emails being sent have a moderately high spam score. This is mostly because the system is intended for a lot more stuff to be in the emails than I want to include, and thus they go out with a lot of blank lines in the formatting. Lots of blank lines are suspicious to spam filtering algorithms. So I would ask any who like receiving the updates-via-email to add *@starshiptim.com to the whitelist in your spam filter of choice to prevent the emails from going into your junk folders unseen. If you don't know how to do that, just ask me, I'll walk you through it.
- I have always disliked WordPress as a platform, and these days, while I don't exactly hate it with the fire of a thousand suns, I heavily discourage anyone from using it unless there are mitigating circumstances of some sort. This site, obviously, has nothing to do with that platform and never will, and my reasons for eschewing it are many. One of them is that the WordPress platform has become ubiquitous, it's everywhere, and thus bad actors—hackers, phishers, malicious billionaires, etc.—target WordPress sites specifically to do their fuckery. They target other sites too, of course, but there's a reason an entire subindustry of WordPress repair and protection services has popped up over the years. Anyway, with that in mind it should not have surprised me to find in the data from my recent experiments in bot-fighting that many of the malicious bots attacking this site are specifically trying to find WordPress login pages and file structures. Shouldn't have, but it did. In a way, it's comforting—it reinforces the belief that bots and their operators don't do subtlety. They're kind of like Rob Deer, really; brute force, swing hard and either get the homer or strike out. So, another thing to cite in my ever-present recommendation that if you use WordPress you should move to something else (I'll help you, my rates are good!), and if you're thinking about starting a WP site to think again.
Even more tweaks
In my continued war against the AI robots, I have been pleased to see one element of the Cloudfare service becoming well worth the effort of connecting with it: The "AI Labyrinth."
For a while I was getting annoyed at the fact that certain bots that I had successfully prevented from accessing stuff here had, according to my standard traffic logs, come back after linking up with Cloudflare. That's the opposite of what I was trying to accomplish, after all. But I did a bit of cross-referencing with all the new data and found that while, yes, those bots were being allowed back in, they were lured in only to be hooked by a sneaky line and pulled into a trap off-site. The labyrinth is a swarm of nonsense and links to garbage designed to keep the scraper bots busy scarfing up completely useless crap while they go in circles, thus wasting their resources. Karma, you bot bastards!
It took a little while for the labyrinth to kick in on this site, as it's relatively puny and not heavily trafficked. But I implemented it elsewhere on a couple other sites today and damned if it didn't start kicking in within an hour. Those poor saps were being inundated and it took no time at all for the lures to get bites.
I had (very) briefly considered setting up my own bot trap, but aside from inflicting annoyance on the bot operators, it wouldn't help; hosting a bot trap is self-defeating as it by design consumes a lot of bandwidth. So I'm happy to be using someone else's likely far more effective one.
So, positive developments on that front.
Not so positive on the RSS feed/email subscription front, though; having had to leave the perfectly fine email program I'd been using because Cloudfare screwed with it, I now have a digest email thing going via "mailer lite." It is playing very nicely with all the security tweaks here, so that's good. It also sent out a duplicate email today, which is not good. That's annoying and starts to get into spam territory, which is no good to anyone. (Apologies to those of you who received it.) I would recommend using the feedrabbit service I mentioned on Sunday, but after making the necessary DNS changes to accommodate Cloudfare, feedrabbit was hit with the entire RSS feed of the site going back years and went a little bananas with its emails. That triggered an anti-spam thing that made it too switch to only sending out a daily digest rather than an update whenever there's a new post. So... oops.
I'm working on making this new thing work the way it's supposed to and not send duplicate mail. Your patience is appreciated.
Meanwhile, as always, I recommend setting up your own RSS feed reader or browser plugin or Outlook folder or whathaveyou to follow not just this site but any others you like. Then you won't need to sign up here. But I know people won't do that, so I plod on with the email thing.
No Comments yetMore tweaks
The battle against the bots goes on, and I may have learned enough about the Cloudfare service to make it actually worth something. Unfortunately, doing so gives it some problems when it comes to playing nice with others, and the casualty of the moment is the email updates some of you rely on. So, as another experiment, I am transferring those email updates to a sort of newsletter format using a third-party service. It's not ideal, but it does seem to get around the Cloudfare interference problem.
So, for now anyway, instead of getting an email whenever there's a new post, those of you who are already subscribed will get an email once a day with the latest posts. Hopefully that'll work out, if not, well, more tweakage to come.
This doesn't affect anyone who's signed up via the handy "Get Email Updates" button at the top right here. Those are through the no-frills Feedrabbit service and still work fine. So if you prefer the one-per-new-post email over the once-a-day digest emails, I suggest signing up through that and then letting me know you want to stop getting the digest versions. I can't shift you over to Feedrabbit, that's got to be your own doing.
Thanks for your continued patience. Damned robots.
No Comments yetTesting 1,2,3...
If you're reading this, congratulations—you have successfully navigated the login page.
Like I said in the earlier post on this subject, I'm not wild about forcing people to log in to see this website, but I've got to find some way to curtail the scraperbots. They just keep coming, and the Cloudfare experiment wasn't mitigating bandwidth usage even though it appeared for a while like it might be. There was some positive effect with it in other ways, but there was also some negative and on balance it doesn't seem to be worth it. Most of what it was blocking were things I was already successfully blocking without it, the only real difference being I had a lot more data about the attempts.
So, now I'm trying this. I hope it isn't too much of a headache for users; it should only be a once-a-month thing for you all on each device you might use, just be sure to check the "keep me signed in for 30 days" box, or else you'll be prompted to log in every time on every page. That would be seriously annoying.
Of course, if my low-tech method of giving login creds doesn't fool the bots, this will be for naught and I'll end the experiment. We'll see how it goes.
UPDATE: Too many functionality problems with this, so I'm killing it already. The experiment lasted less than two hours. It was successful in blocking the bots; that worked perfectly, at least in the short term. But it also prevented RSS feed operations and link sharing to social media and various other positive elements. If I ever find a way around that, then maybe I'll try again.
No Comments yetExperiment insights
The bot invasion continues, but my newest experiments in combat tactics are at least revealing some information. The Cloudfare service I'm trying out is a mixed bag, to be sure, and I'm still not sold on it being all that useful, but it is able to see through some of the bot camouflage. That in and of itself might be helpful in creating IP blocks that actually work. We'll see.
For example, there's a hit in my access logs from about an hour ago ostensibly from Senegal on a certain IP, using Chrome version 103 on Windows 10. It's obviously a bot because it matches certain patterns in my access logs, but it successfully oozed through all the "bots keep out" forcefields and convinced servers it was a human user. The extra layer of filtering reveals that this bot was actually in Venezuela on a different IP, using Chrome 110 (headless, which means no user interface, which means entirely under-the-hood snooping), using or simulating Windows 10. Another hit claimed to be from Mexico on a certain IP and was really from El Salvador on another, pretending to be on a MacOS.
It does appear to be intercepting a lot of bot traffic, but I'm still a bit unclear on what's being reported; a lot of these, I think, are bots not trying to disguise themselves (like Googlebot) and thus hit my robots.txt file and see the "keep out" sign and obey it. These would be stopped anyhow, I'm just seeing more data on it this way. It's also reporting more bandwidth usage than my actual usage seems to be, which hopefully indicates intercepted bots that don't make it through the filter? It's going to take a few more days worth of data to give me any sense of difference its making in terms of stopping bots in their tracks.
Meanwhile, I am also considering turning this site into a "members only" sort of thing that requires a login for anyone to see it. I don't like the idea conceptually, but password-blocking is a sure way to eliminate bot theft. At the moment, I can only assign members manually, from my admin interface, so that's a non-starter. I would need to code a new feature into the site guts to allow people to sign up from outside, and then add stuff to prevent bots from signing up (or at least minimize them). I may have a low-tech workaround for that, based on the idea that bots, especially these newfangled scraper bots, will look for more technical means of defense to thwart and might miss a simpler approach.
How would people feel about being forced to log in? I well understand that the average websurfer has the attention span of a gnat, and that any obstacle might be enough to cause them to move along to something else, but this seems like a simple enough ask. Any thoughts?
I'll try and implement that simple workaround login test tomorrow/later in the week. Can't start on it now or I'll end up losing track of time and I have to be out of here in another hour or so because I start my 2026 umpiring this evening. It'll be cold, the field will be wet, and the teams playing don't include any of my favorites (though no serial troublemakers, either). I'm not exactly excited about it, but it will be nice to be back on the field again.
No Comments yetBot escalation
Futurama's Robot Mafia
The neverending battle continues.
A couple of months ago or so I had landed on a potential mitigating tactic to use against the onslaught of so-called "AI" bots monopolizing the bandwidth here aboard StarshipTim.com. It worked for a while, but those bastards, just like the Borg, adapted, and now the problem is worse than ever.
I am at my wits' end. These (presumably) scraper bots have managed to circumvent all barriers against them by convincingly camouflaging themselves as human users and rarely/never using the same IP address twice. Blocking them has become impractical without taking drastic measures.
In and of themselves, these bot visits aren't debilitating. They are most likely illegal in the sense that the scrapers scouring the Internet to feed large-language-model algorithms don't give a tinker's damn about copyright law, but in practical terms, what's to be done about that? At some point there may be a class action suit or several to take part in, but proving theft is tricky and the operators of these bots know it. But each visit isn't a drain on resources by itself. Cumulatively, however...
Since the calendar turned to 2026, there have been 203 visits to StarshipTim.com, not counting hits from myself when posting and checking comments. 185 of them are from unwelcome bots. Nearly 2% of the bandwidth this account is allotted for the month has been used already by bots. If that usage rate stays consistent, 12-15% of my bandwidth for the month will have been stolen by bots circumventing the "keep out" signs. This is a rather low-trafficked site, so that won't hurt me here, but really there's no reason to think the bots will keep themselves to this pace.
I host other sites on this server too, and the larger/more-trafficked of those are hit much harder. On one such site just one percent of visits today were (probably) human users, the other 99% were unwanted bots stealing bandwidth (and stealing content for their LLMs). Gigabytes of bandwidth have been used on that account since 12:00:01am January 1st, almost all of it bot traffic. Typically, that site will use about 30GB per month; last month it used 110GB (mostly bots), this month it's already on pace to use about 150GB, far exceeding its allowed usage. I've been borrowing bandwidth from other accounts to cover the excess so that one doesn't go offline for violations. The client isn't at fault. Elon Musk and his ilk with their scraperbots are.
Unsurprisingly, there is an industry starting to sprout around fighting the bots. This abuse of the system has given rise to a profit opportunity for companies that will basically envelope a site into its own bubble and charge thousands of dollars a month to mitigate the bot traffic. Useful, I guess, for giant corporate websites where adding a several-thousand-dollar-per-month expense beats the loss of resource to bots expense. Cloudfare at least offers something more affordable for the little guys, $20 and $200 per month services that attempt to filter bots by routing your visitors through their server security first, then sending those that clear to your own server. I wonder about the false-positive rate with such a thing.
This is maddening because it's a new example of our societal profiteering on bad behavior. Take a walk around the neighborhood, see how many homes have ADT or some other security service's sign in the yard/window/whatever. These people pay every month for protection against bad behavior. Nice home you got there; be a shame of something happened to it. Now imagine that a bunch of Techbros were running a systematic campaign of stealth home invasions that copied and photographed all of your possessions while running up your electricity and water bills, and that these invasions happened every day, usually more than once. Now further imagine that law-enforcement didn't care about these invasions, that lawmakers were browbeaten into thinking that these invasions were somehow permissible because they involved a digital computer program that allows for entry into the home, so technically it isn't breaking-and-entering, and none of your stuff is actually missing when you get home, so is it really theft? With all that going on ADT would be set to make a fortune in new subscribers, right? Symbiosis.
I'm actually experimenting with one of these pay services, the smallest scale option, to see if it makes a lick of difference. If it does, then I can offer it to clients as a mitigator, but I don't like it. It feels like letting the terrorists win.
1 CommentThere's an app for that
I spent some time this week doing some maintenance on this here website and found myself going down a bit of a rabbit hole regarding optimizing for mobile screens, so much so that I made an Android app for the site. So if you have an Android phone, congratulations, you may run the new StarshipTim app on it.
It's not special, really. Basically it's the site as it is if you open it on your phone in Chrome. But it's got its own little icon to tap and open up without having to navigate around your browser.
Basically, I made it for my dad since he can never get to the site without google-searching for it first. But he might not even use it because you have to install it manually; it's not like I'm going to pay to put it in the GooglePlay store. Screw that.
But if anyone wants to give it a go, download this file to your Android phone: Starship.apk
Once it's in your downloads, open the Files app, go to Downloads if you're not already there, tap the three-dot icon for a menu and select "install." You may then have to give permission to install apps from places not controlled by Google, but just do it, it's your phone, you don't need Google's permission. It may then try to scan the app for nefarious code, that's fine, it's just an annoying delay of a few seconds. That's it, you've got your app. It'll show up in your Apps list and you can add the icon to your homescreen shortcuts if you want to.
You'd think my doing this means I now know how to do Android-specific code, but no, not really, I had a hell of a time getting this to work at all and this is, like, rudimentary. Android Studio is not exactly intuitive if you're just flying by the seat of your pants. In the end I used a third-party conversion tool instead.
Apple being Apple, iOS apps are more difficult/expensive to make, so you iPhone folks are hosed. Sorry. Blame Tim Cook.
No Comments yetThe Cylons were created by man
Elon Musk
The war against the bots rages on.
For this website, as well as others I host, I have attempted to install security measures to cut down on various types of bot traffic. Some have worked pretty well; spam comments are way down, for instance. But the newest wave of Internet bots is more tenacious, ruthless, and borg-like. They adapt.
These are scraper bots. Automated scripts that bypass traditional filters meant to regulate non-human user traffic in order to scan everything written on a web page and plug it into a huge repository called a Large Language Model to train so-called "artificial intelligence" (a misnomer as it is neither) programs. These scraper bots spoof their identifying markers so they appear to be a regular user using a regular web browser on a regular computer, but they're not. In Battlestar Galactica metaphor, they're skinjobs—they may look like human beings but they're still Cylons.
Thus far there seems to be no way to adequately combat these plagiarism factories without either adding whole layers of expensive third-party software firewalls or forcing every human user to log in with password credentials. I've tried blocking the bot IP addresses, they just cycle through new ones. I managed to eliminate a lot of them by blocking all browsers using Chinese, but then within a day or two they were back using English. I've tried blocking their spoofed configurations—generally they prefer to show as a MacOS using an outdated version of Chrome with an obsolete screen resolution—but that only nails a small fraction of them since most don't really use such configs; those are the fake IDs shown inside the bar, not the different fake IDs used to get past the bouncer.
My latest attempt at blocking them, which I will not explain here, appears to be effective for the moment. No bot traffic for several hours now. But like any good Borg drones, I rather expect to check the logs tomorrow and find that they've adapted.
Fortunately, these bots don't use up a ton of resources; since they don't actually render the site on a browser, the bandwidth usage on each hit is relatively small. But it adds up. And they're everywhere—estimates are that over 50% of web traffic today is bots and that as much as 80% of that is "AI" scrapers. Other estimates are less specific, but measure over a third of all web traffic as "bad bots," i.e. malicious actors of one sort or another, but whether or not "AI" scrapers qualify as "bad" depends on who's doing the study.
I blame Elon and Zuck, but let's face it, if it wasn't them it'd be some other entitled asshats thinking they can just do what they want and steal everyone's work with impunity.
I'll now wait and see if my latest mitigation is worth anything, and if so start applying it to client sites.
1 CommentTechnical difficulties
I've been spending a ridiculous amount of time lately trying to police the traffic on this here website from robots. There's a new generation of crawler bots online, ones that ignore directives from robots.txt files and successfully masquerade as a human user, and they've become the majority of my traffic here, sucking up resources.
Granted, not a heck of a lot of resources, they're not interfering with any actual humans being able to access things here. But they're annoying. And, more importantly, I don't know what they're doing.
Best guess is that they're scrapers, looking for email addresses or other things in the text of websites that will facilitate marketing/spamming/nuisance assholery. Secondary guess is that they're bots sucking up text to use in building so-called AI large language models. Which is, at its core, copyright infringement.
Anyway, nothing has worked to block the bots. They get around everything. They avoid the bot blocks by spoofing a browser signature, so I block the version of the browser they pretend to use. That fails, because they're not really using it. I block the IP address range, but they just VPN their way to new ones.
It's pissing me off. But I'm also out of ideas, at least for the moment.
In the course of trying various block strategies, I broke the RSS feed. So for the less-than-one-percent of you that use the feed in Outlook or a browser RSS plugin, and for the few of you that rely on email updates (which are based on the RSS feed), you may have encountered some wonkiness over the past couple of days. Sorry about that. It's fixed now.
If I could just find a fix for the damn bots.
No Comments yet
Mental lapse of the week
Hi. We're back. You may not have noticed we were gone, but for a while tonight StarshipTim.com was dead in space.
It was, of course, related to my upgrade endeavors, and likely because I had expected there to be some kind of problem or other, it took a long time to solve the issue because it was an unbelievably simple matter, which is to say, nothing was wrong.
I had uploaded the new guts, upgraded the PHP on the site, transferred the database, and yet, nothing but error messages. Now, I could blame the error messages for being way too vague, but I was befuddled enough that I did what no coder likes to do: ask for help from tech support. And tech support at the server farm fixed things by...doing the thing that I had just forgotten to do after loading the database, which was to flush the cache from the old database.
It's something I did literally dozens of times while building the new guts. I just forgot to do it when I tried to take it live.
Which, fine, we all forget things, but it's embarrassing to have gone to tech support and then have them come back with the PHP/MySql equivalent of "you need to turn it off and turn it back on again." The voice of Chris O'Dowd is mocking me in my head repeatedly.
That said, now that it's been done, I think all is working and we're on PHP 8.1.
One bit that I haven't yet conquered is the functionality of emailing me when someone posts a new comment, but I decided that doesn't matter because it's redundant to the comments RSS feed and I can just email subscribe to that feed.
Seriously, RSS feeds are massively undervalued by Internet users everywhere. The oligarchs (get used to that word, folks) don't like them because they allow the end user too much control and allow for easy ways to get the content you want to see without bothering with their platforms that they rely on to datamine the shit out of everyone in the world (also, because they can be structured to serve you an entire article without showing the advertising on a website, but really it's the datamining).
No Comments yet



