I really appreciate the Archive team's efforts to make the Wayback Machine more responsive, and have donated a few times to support them.
Unfortunately, the restrictions have been way too strict for the last few months: from my residential IP, simply moving the mouse on the calendar for a specific URL is enough to get stuck on 429 error messages for a while; and from corporate ISPs (for example, on airport WiFi), you often can't access the WM at all. I hope they'll find a way to relax those.
Generally speaking I feel like detecting the bots might be a lost cause. For someone like the Internet Archive I don't know how to deal with it, for smaller sites, cache everything, static pages whenever possible.
Sadly I see rate-limiting usage in general becoming a thing. With residential proxies and more sophisticated bots either pretending to be Chrome or directly piloting Chrome, it's going to become impossible to tell a real user from a bot. Only solution is to pretend that everyone is a bot and design for it.
we’ve been using Datadome at work for this because it’s nice to get someone else to think about the constant bot cat and mouse, and we all benefit from rules and fixes created from other client data. Not an ad- that service is eye watering expensive but I think it makes sense as something to offload.
We use spur.us which provides us with information on IPs they believe is running proxies. It's a nice service, in terms of pricing it's completely reasonable, for our use case. I can't imagine how much work goes into compiling their data.
I have had similar experiences just opening archived pages with a couple of embedded images. It's at a level where just using the site normally is painful.
Reddit's ad platform is the worst imo. I pushed for trying it at my work and I ended up looking like a fool: it blew all the budget almost instantly on phantom clicks, with absolutely zero conversions. It's nowhere near as good as Meta's.
Prime's app is ridiculously bad imo. I'm not sure why that is, but everything is slower and takes more clicks and digging than similar apps on the same device. There are also significantly more ads than on Disney+ or Netflix ad tiers.
I find that Netflix and Paramount+ are the American services that offer the best app experience on my (very low-end) smart TV.
There is a very simple explanation for why weaker models appear to kick sand in Fable's face: Fable cannot be benchmarked because of its batshit out-of-control refusal policy.
If it actually tackled all of the problems it was assigned, it would presumably kick Opus into the weeds.
Do you have any tips for running Codex on the Macbook Neo? Have you changed any setting? On mine, it stutters on every single chat, and I don't even have any plugins installed.
Are you on the recent update that handles long chat sessions much better? That made a big difference. Biasing towards lightweight and fast tool alternatives (uv over pip, bun over npm, ripgrep over grep, etc).
If that still doesn't resolve it, a free tier VM might be the move. I suspect OpenAI will be bundling some kind of VM with their new Astra release, which Sama said yesterday will be released soon (this week maybe?). If this new model supports much longer running tasks, you'll want a dedicated box that can run 24/7 even when your Neo is closed and asleep anyway.
I had the same problem, tried a bunch of stuff, banned millions of IPs, but nothing really worked. I ended up spending a bunch of time rewriting the website with Codex so that it is more efficient. Now I'm still getting the same junk traffic, but it barely affects the CPU anymore. I think this might be the way to go.
Unfortunately, the restrictions have been way too strict for the last few months: from my residential IP, simply moving the mouse on the calendar for a specific URL is enough to get stuck on 429 error messages for a while; and from corporate ISPs (for example, on airport WiFi), you often can't access the WM at all. I hope they'll find a way to relax those.
reply