> No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
We can and have put some ugly, hissing, spitting cats into bags, some of which weren't even in one to begin with:
* Chattel slavery is illegal, and the vast, vast majority of humans agree it's appalling it ever wasn't.
* Food manufacturers in the US are no longer allowed to put highly-dangerous additives to their products and hide the fact.
* Women are allowed to vote.
* Ten-year-old children are no longer being maimed and dying in American factories.
Claiming it can't be done is either fatalism or propaganda for execrable companies whose evil has rarely been matched in the history of capitalist business.
Do I have a clear solution for how to stop the insanity? No, not yet, but that is not a proof there's no way to do it.
You can of course choose how to spend your time in this world, but I don't see the value of either 1) assuming we're screwed and giving up before trying to fight, or 2) shilling for lying assholes like Sam Altman.
Thank you for this response, and I will clarify. I would roughly lump all of those into "social issues," whereas I would put "using AI to advance math" under "technological progress." That's why one article likened it to Socrates's fear of the written word. In broad strokes, I would say that once a technology becomes available, no amount of social pressure will stop it from proliferating.
If there's a distinction being made between "using AI for math" and "the particular methods being employed by OpenAI", I think that's splitting hairs. The core of the issue is that math is being done outside of people's heads by a different thinking system and they're worried about the implications.
If you look at something like the EPA (or the FDA as you mention), it is true that certain uses and side-effects of technology have been identified to be detrimental to society and have been restricted, but this letter doesn't seem to be in that league, it really seems like sour grapes to me. Dumping toxic chemicals into the water table is one thing, saying "we don't think people will learn from these methods in the same way that they traditionally have" doesn't hold as much water to me.
To expand further, I think it's interesting that both replies to my comments in this thread sort of alluded to the idea that I might be a corporate shill or at best a patsy. I feel like that couldn't be farther from the truth, I walked away from the software industry and corporate world years ago.
Perhaps I'm looking at this from a too zoomed-out or detached perspective. I'm just thinking about the concept of the technology at play and its future impacts regardless of who owns or controls them. But maybe that's naive. In today's connected world you can't separate those things, and I think I'm understanding that a lot of the outrage here is more about the socioeconomic implications of a big corporation suddenly "disrupting" a field that was largely owned by more altruistic academic institutions.
I do still think that the internet, despite largely being owned and controlled by the powers that be, has represented a great shift of power and agency to the individual, and is ultimately a socialist construct, and we will continue to see those changes over time, and that AI is an extension of that.
To expand EVEN further, look at the history of the written word. Being a scribe was a high class position in service of the church. The printing press was invented to print the bible. These were the information-theory projects of the hegemonic institutions and propagandists of the day. I don't think anyone today would argue that these developments ultimately benefitted and liberated the people.
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
> I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
> The perverse incentives of academia mean this has never occurred.
This. Mathematicians in their most energetic years are trying to get tenure or land a tenure-track job. They are disincentivized to go all-in on ultra high risk, high-reward problems. The potential downside is just too forbidding. It's much safer to develop a research program in a mainstream field that affords many opportunities for partial progress that can translate to a robust publication record.
"I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already"
There were more than six doing that and it's essentially why it was ripe for AI to finish it off. But the finishing off was quicker than anyone expected
I'm not a mathematician, nor do I know almost anything about the discipline, but it sounds like maybe you do.
What teams of people spent an entire career working together, focused entirely on Navier-Stokes or problems they suspected were related, without any "publish or perish" concerns?
I realize tenure is a thing, but my limited understanding is that a significant amount of time is spent earning it, once you account for Ph.D. program and the years of needed to be granted it.
ok I realize this is a tangent but you're saying my post is very misleading and then also saying that a human mathematician's career is about 100,000 hours of compute and that Navier-Stokes could've had a solution by now if not for perverse incentives. You may be right but I don't think this is a great argument because in a year I would bet that those numbers change since computing power tends to increase or get cheaper over time. So I am taking the stance AI can outdo people if not now, perhaps soon.
I wasn't trying to say that genAI won't beat humans. It arguably already has, much as that may fill me with horror and revulsion.
I'm just trying to point out that economically, we have not yet reached the point where AI mathematics research is a no-brainer hands-down win, no consideration required.
It might already be a win, and certainly the ability to compress those 900,000 hours of effort into an actual week of linear time is mind-boggling and potentially a huge game-changer for all kinds of open research questions.
It's not obvious that human math research is dead yet.
I would care if even these machines could only write articles, and I cared when software destroyed multiple forms of media delivery.
I won't say it was all bad, unequivocally, but it made me sad and I tried to think of solutions.
Turns out, alas, that make people are willing to trade away freedom, independence, privacy, and a functioning culture to save a few bucks and get dopamine hits on demand.
And then after a few years the moralists have to fight a world-spanning war to keep the pragmatists from completely eliminating people groups they don't like from the planet.
Sampling in music is long settled in the US - it is a copyright violation if not licensed, with the rare exception being cases where the sample is fair use:
I suspect people who are putting out LLM "writing" are people who weren't trying to communicate in the first place.
They were likely hunting for a better job, chasing the thrill of a million views, or similar - seeking a side effect of writing, rather than seeking to help someone else understand what was in their head.
The thing I trust the most to solve tricky problems reliably is a specific very skilled programmer I've known for twenty years.
I wouldn't say he never makes mistakes, but his success rate is a damn sight better than any LLM I've ever interacted with (and I drive Opus daily, due to corporate demands to use LLMs).
Same partial answer as I gave your sibling commenter:
"OK, so the task ends up being to write a 40-page analysis of the result of a specific experiment in quantum chromodynamics, to be finished within 2 minutes. Does your choice for a human work, or would you have rather chosen any of the frontier AI assistants? Be very, very honest. Remember that the lives of your family are on the line and nobody will judge you for a lack of allegiance to humans."
This task is intentionally designed to ensure a human cannot do it.
The initial scenario is utterly, insanely absurd to begin with, but I tried to go along in good faith and gave you the true answer.
The result was a bad-faith rhetorical trap, so I'm done with this thread.
In another attempt at good faith, as part of bowing out I will add some actual response to your anti-useful cheap rhetorical trap:
I do not trust LLMs to get things right in high-stakes scenarios. I have seen the current models spit out falsehoods and errors regularly in the handful of fields I have expertise in, and have no reason to think they would do otherwise outside my expertise.
The scenario you describe is an absurd fiction, and no human making the absurd threat could evaluate the paper in less than hours (realistically even an expert would need days, and a nonexpert could not do it at all [short of becoming an expert]).
So, there's no point trusting a bullshit machine to save my family - it might very well get them killed, and whether it was right or not, what would actually matter would not be its correctness, but what the presumable bullshit machine evaluating my offered input spits out.
So, the best move I could realistically make would be to put a stab at prompt injection into the input.
For that job, I probably would actually prefer aforementioned programmer over any other option, come to think of it - I suspect he'd have better success than even another model (especially considering the safeguards the models no doubt have to try to keep users from using the models to inject other models).
Again - I'm disappointed in your worthless rhetorical cheap shot.
I suspect you'll have much better success convincing people LLMs are intelligent if you engage in good faith, listen to their perspective, and address their actual thoughts, instead of devising the sort of inanity that comes out of high school debate clubs, where people literally want to score points instead of find truth.
> This task is intentionally designed to ensure a human cannot do it.
It is one of a vast array of things the hypothetical kidnapper could come up with, some of which humans I agree will do better at (currently) and some of which AI will do better at. We clearly agree that in that array there is at least one task that a frontier AI would be better at than any human you could pick.
It is a thought experiment, so there is nothing fundamentally wrong with it being extreme or unrealistic (thought experiments very often are), but for the sake of goodwill let's 'weaken' it a bit: the AI or human always has an hour to come up with the answer, the question and answer are in their preferred language, and the answer fits on four pages. You can't help them, though; The criminal 'prompts' them. They can use the internet as an informational resource, but they can't communicate/ask for help/post anything (with the spirit of this being: no loophole in letting somebody else do the task or parts of it for them). And of course all subject matter of all complexity is fair game (including but not limited to quantum chromodynamics experiments).
Given that situation, do you think the programmer you mentioned would be more successful than a frontier AI in more than 50% of the possible intelligence tasks?
Edit, addendum: Please, if you can, also let said programmer read this thread and give his opinion on it. It sounds like he would have interesting things to say on this.
At present, we have no idea how to do that, so the answer is still "this is not knowable" in practice.
Perhaps that changes tomorrow, or in a month, or a year from now, but until a theoretically-sound technique for understanding what the weights signify is described and demonstrated to be reliable, my statement remains true.
> so the answer is still "this is not knowable" in practice
No, it’s not. It is unknown. To say it may be unknowable you need a fundamental reason why it may not be knowable.
What lies behind event horizons may be unknowable. We have theoretical reasons to suspect this. What LLMs are doing isn’t well enough understood, theoretically, to even say what is knowable versus unknowable. Just what is known and not.
A method not existing and a method being impossible (or unlikely) to exist are separate concepts. When you say something is unknowable, it should mean it literally cannot be known—route around the question entirely.
Okay, I reread the original post, and you're right.
I had gotten my wires crossed and thought the OP was asking about a specific situation, but it was actually a question about the general pattern.
A specific situation will often pass before any theoretical tools can be found that could possibly answer the question.
For a general pattern, though, you're entirely right - if the tools arose hundreds of millions of years from now, that range of questions becomes answerable, and the information is then knowable.
No, it’s not. Rumsfeld segregated what we know from what we know we know (and vice versa). An unknown (whether known or unknown) may be knowable or unknowable—his framework doesn’t address knowability.
> because an LLM is not a God. It is not an unknowable
I tend to agree with you. This has nothing to do with the Rumsfeld comparison being wrong.
There are like ten sibling replies with a lot of speculation but I'm pretty sure this is the correct answer. I tend to agree with the other commenter we might know someday but we don't know now.
> Looks to me like I can run Claude Code without being able to afford my own datacenter.
Until Anthropic bans you from using their data centers, at which point you cannot run Claude Code at all. Welcome to being a have-not (at least in a world where only genAI-assisted coding is acceptable).
Whether Claude is or is not available via AWS Bedrock is entirely decided by Anthropic, not by you and not by Amazon. And you can't buy the data sets (the REAL power...) from any AI company.
reply