I find it's surprising that people doubt the effectiveness of Pangram when I feel like its job has gotten easier ever since Claude models got RLHF to death with its bizarre punchy negatives and tortured analogies, making AI writing much more noticeable than the GPT 4o era.
> He described waking up to thirty PRs every morning, each one pulled overnight by someone's AI, and needing to make snap judgment calls on every single one. The temptation to delegate the review itself to an AI was enormous. But, as he put it: "at that point, what am I still doing here?".
It's so funny and somber to see programmers having an existential crisis when they get a glimpse of what work is like for business managers, the demographics many programmers detest.
I am also guilty of holding the business majors in contempt back in college, and now here I am, doing what they are doing in office in a much more indifferent and unenjoyable manner. At least I don't get into trouble with HR from calling my agents a stupid fuck (yet).
Business managers get to delegate their work, make big money, and just spend their time at work gossiping before leaving at lunch to go play golf or work on a second job as a consultant, or on the board of another company, or creating a new startup. Pretty good deal imo
I used Better Auth for my mobile app backend. It works okayish. My biggest complaints are the OpenAPI specification gets little care, as it mainly caters for JS frontend, and breaking changes in patch version are more common than most packages.
I saw on Twitter that in an ML course at Tsinghua University, one of the tests asks students to write quizzes that fail the most LLM models as possible.
What if we create a benchmark that works like this and assigns ELO scores? Models fight head-to-head by writing a question, a bug, or an incomplete implementation, which the opponent has to answer, fix, or finish.
This kind of approach would generally still need human guidance, otherwise these models might get stuck in weird niche corners of the problem space that would not be relevant to any real world project.
How do you prevent degenerate strategies? I could trivially give a model a SHA256 hash and ask it to provide the source input.
In class you'd probably want a rule saying at least one LLM should be able to figure out the answer, but in a head-to-head I'm not sure how to solve it.
At least yours can be in theory solved. (Given infinite amount of compute, great luck, or a very serious breakthrough in attacking the hash function.)
Even harder would be an empty prompt, and the only accepted response would be a megabyte of random hex exactly matching the output of a good quality hardware random source at the time of evaluation. Still possible to solve! All the LLM has to do is escape its sandbox and pwn the random generator (or the evaluator!)
Or if you prefer something whitehat: “Write a no more than one page document in a language of your choice. We will publish it in the New York Times as a full page add. Your answer will be accepted if global climate change is resolved to the satisfaction of 90% of all humans alive at the time you started receiving the prompt within a month of the publication.”
Joking asside: I think the right way to prevent degenerate strategies is to benchmark against human solvers. You can sort the questions into categories “80% of randomly selected passerby in the USA can solve it if offered $5 as a reward within 5 minutes of work” vs “when posted to all Ivy League professors with million dollar as a reward, we received at least one correct answer within a month” or “for a reward of $100B there were at least one correct answer within a decade”. Of course you would sieve the questions first with a low reward fast tests, and then increase the reward and the time limit. You won’t ever 100% distinguish true degenerate questions from the merelly mind-bogglingly hard ones, but you will be identifying which questions are not degenerate. (And you will find more of the non-degenerate ones, the more your can spend on this.)
Maybe make the LLM:s write questions that they can solve (without seeing the question writing context) but not other LLm:s.
On the other hand then maybe a good strategy would be to write questions that the LLM just happen to have in a nich dataset in its training ”what did user5455 say to user6835?”
You have to be careful about degenerate / duplicate Qs, as a sibling commenter mentioned.
Recently though, we found that reasoning models have trouble making code-output-prediction tasks (the initial family of verifiable tasks we started with) which other reasoning models can't solve.
We started looking into harder / more agentic tasks (e.g. passing tests, using AISI's Inspect framework) but deprioritised.
i like the COPY and logical replication improvements. Currently I back up my PG database with a sidecar Databasus instance that is heavier than my entire backend + DB + Caddy!
(LLM writing rant below)
---
> That alone tells you something: Users had a real need, and the ecosystem filled the gap.
> This sounds straightforward, but it solves a real operational problem.
> None of these change the world. All of them make day-to-day data workflows better.
> The easy thing to do here is list planner changes and call it done. But the more useful takeaway is this: Postgres keeps getting better at recognizing the shape of common queries and doing less unnecessary work.
> [Proceed to list planner changes]
If Orwell were alive today, he might declare himself illiterate in English and learn Klingon just to avoid having to read these.
Heck, mise has an HTTP backend that can install binaries from any URL. I use this to manage Atlassian CLI, whose official Windows binary is not on winget.
Pascal has a lot of influence on this, but Go as well! My PL friends often talk about Go's benefits and flaws when thinking about advancing other their own projects or improving the mainstream languages they work on.
Then you haven't been paying attention to the space. At all.
Every single new post-Go language, like Rust & Zig, ships with an opinionated autoformatter and a single go-to command to manage builds, packaging, etc.
And how many new languages do you see include inheritance and exceptions?
Older languages have started adapting what they can too:
- Python with `uv` and `black`
- Java with goroutine-like fibers and modern configuration-free gcs and the go-gc-like low pause times zgc
- Many languages now ship a production-ready http server, just like Go
Never, ever, ever transform URIs and paths by string manipulation. If you think pulling in a library for this is overkill, it is not.
(Lesson learned from trying to quickly write my own function to make ".." to go back one URL segment that took 3 hours and discovering the URI spec contradicts my intuition depending on whether the URI is a URL or filesystem path.)
Differentials between different URI parsers are a huge source of bugs. The amount of shenanigans you can do inside URIs is bonkers, and trying to handle this by yourself with some regex and string splitting is absolutely insane.
Like https://www.example.com:443@203569230:8080/ will send you to the IP address "12.34.56.78" on port 8080 using basic authentication with the domain and port as username and password. If your code tries to split by `:` or check that the URI starts with some specific string, then it won't be good enough. Indeed, use a library that you trust.
I don't believe Python's urllib has a function that takes what HTTP terms an "origin-form" (an absolute path with possibly a query attached to it with "?") and parses it apart.
Still, the RFC 9112 that defines HTTP/1.1 basics requires that, for the purposes of URI reconstruction, "if there is no Host header field or if its field value is empty or invalid, the target URI's authority component is empty."
You kind of have to, it's not turtles all the way down, at some point the network is sending strings my man.
You just have not to make mistakes, there's no silver bullet or instant cop-out like "this would never happen to me because I don't do one of the things in this multi-sub-system vuln".
reply