The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says.
Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic.
He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. Please at least consider the possibility.
This sounds to me like a cry for help from someone thats held hostage.
His previous self is writting this from the possition of his current self who is too deep in the economic consequeces to be able to do anything meaningful other than write a consequenceless text. Any other action would now carry too much personal risk for him.
You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?
AI technology is now only audited by people who think like them. The ones that dont either dont join the company, or leave after a short stint as we have seen. That creates an information or feedback bubble which is not healthy nor productive.
Okay, but suppose that Hypothetical Opensource Anthropic trains a model that turns out to be very dangerous, and releases it. Suppose that the public investigates and, not being limited by an information bubble, correctly notices that it's very dangerous. What then? The model's already released, there's effectively no way to prevent it from being used. Whatever the risks of its release were, they will now materialize, regardless of what the public wants. How is this better than the current world, either by Dario's values or by yours?
By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic. At least an open source anthropic would allow to join economic forces to try to combat that situation, for example. But there are other more clear, less distopian benefits.
> By the same logic, suppose that Dario/Altman/Jensen accumulate all capital because they are the only one that have access to AGI and end up controlling democracy, turning the world into technofeudalism or whatever you want to call it. How is that better than an opensource Anthropic.
It'd be pretty bad but it's also a world where humanity lives on, which is better than a lot of other outcomes. A world where everyone has unrestricted access to AGI is like a world in which everyone has a tactical nuke in their pocket. If somehow AGI doesn't lead to x-risks, we will "merely" have to survive in a world where rogue agents can do whatever they want and defenders can only ever react. Technofeudalism would be bad, but this (technoanarchism?) is quite bad too.
However, neither does Dario seem to propose to become world dictator. Like, I'm sure he wouldn't say if he wanted it, but notice that he isn't particularly trying to aim for that outcome, either. The plan in this essay involves third-party oversight and federal control and global cooperation - IMO it's about as non-dystopian an outcome as we can hope for, if we build AGI at all.
I agree. Dario is a very smart guy and he is probably aiming for the best solution for humanity that at the same time keeps VCs/whoever gives him money happy, and keeps him in control because he does not trust the rest (probably with good reasons).
My point is more that he is aiming for a local optimum for society, but he could find a lower optimum if he was not trapped in the path that he chose early on.
I dont buy the tactical nukes/end of the world narrative. Why is he not speaking about the loss of cognitive skills of the population for example? That is a more real danger that is starting to happen. There are articles already speaking about the use of LLMs as cognitive viruses. Why are his aligment teams not reasearching and publishing about this?
Why don't you create your own auditing org to fix this? Or at the very least, publish a detailed critique of what you believe existing auditing orgs are missing.
Will they give my auditing org access to their proprietary confidential secrets? Or will that happen only if they know i agree enough with them so that i am not a risk?
> "Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy."
Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.
If he believes what he's saying, then when Anthropic is sued for their AI harming another party, his statements are evidence that Anthropic knew _in advance_ that their AI safeguards were likely insufficient to keep their product from harming people. It would be a blatant admission that they were reckless and negligent. That other AI companies are doing the same would not mitigate that.
> it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.
not really. if you really want to go by definition in the book and apply it the CEO role, Amodei is CEO of an LTBT, so it's in the CEO's goal to benefit humanity. You can believe in his sincerity or not if you want, but using the CEO label as a definitional reason as to why he must lie is factually incorrect.
Not sure you can judge a PBC under those same umbrella as a for profit company, and voting rights make a big difference in terms of responsibility. There are a number of ways to govern companies that can reduce the amount of cynicism and “shareholder value” issues - you just don’t see it very often.
> Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation.
This seems like useless pedantry. Suppose a person really does have belief A and expresses it for years, they become a CEO of a company, and keep saying A. Are they lying? Well, maybe their beliefs magically changed when they became a CEO, but the more likely explanation is that they think expressing their belief in A is more important than their company's interests.
I have considered it, I've been hearing way more of it that I like and it's rationalist slop with little to no predictive power: https://foom.hyperplex.org/
Hmm, I clicked that link and the first claimed bad prediction I see, from 1996, is "singularity 2035 (actually 2025)".
Predicting 2025-2035 as, at least, the period when AI becomes a really big deal, seems pretty good, even if the jury's still out on "singularity".
Broadly, the rationalists seem to have been pretty early to realizing LLMs were a big deal, and certainly seem to have had a much more accurate picture of how they'd develop than the people who denounce "TESCREAL" and talk about "stochastic parrots". It's fair (and IMHO correct) to ding rationalists for lots of things, but specifically poor prediction about AI seems like a bad one, insofar as anything has been tested so far.
It's more like we just invented ball bearings. We just jumped from standard to industrial grade, and precision grade is on the horizon. All kinds of new possibilities have opened up, cars can go a mile a minute on these things! Surely if we keep increasing the precision at this rate, we'll defeat friction once and for all.
Gemini models - at least via some interfaces - have tool calling API access to various Google integrations. flights.google.com, maps.google.com, etc.
The info isn't in the model weights.
Because of where I live, there are three viable airports for any given flight I might want to take, which historically has made shopping a real pain. But Gemini (and only Gemini) has greatly simplified it. Pramble plus date range plus destination and it very quickly generates potential itineraries with costs, total travel time (driving included), etc.
Why not set a global instruction that their direct outputs to you should be in your native language?
For a long time I had Claudes (in the 4.0-4.5.x range) use only French in the chat, while keeping English for working docs (and the code, obviously). Works just fine.
edit: I can guess that any right-to-left languages would likely break claude-code rendering?
Before OpenAI / situational-awareness he made a lot of money on societal-shift type investments and shorts during the lead up to Covid economic impacts.
His "main thing" is success in calling economic impacts of undervalued large shifts, and the premise of the fund is basically that the same thing is occurring around AI, where he also has specific subject matter expertise.
Well, I don't buy it. There's probably a lot of people who picked out those trends, after all they are just what's in the news. Most of them didn't get 45B to invest.
I'm going to need better evidence to believe he has any skill.
From what I can see, his investors could have just bought AI related stocks themselves.
That depends on how long the trend lasts for. You could have got into stocks like Apple, Amazon or NVidia very "late" and still made tons of money.
Aschenbrenner only created his Situational Awareness fund 2 years ago, so he wasn't exactly prescient in predicting the rise of AI - he just had enough conviction to go all in, with leverage, on an investing theme that was already pretty obvious.
Years ago I also did some experimentation w/ midi-device and SRS ( https://www.youtube.com/watch?v=a6tvHMvF8Mo ), where the focus was on ear-training rather than score-learning.
Clef seems to be a pretty strong attempt at a high difficulty UX. I've created an account and will be giving it a go. Wishing you luck, and thanks for sharing.
A feature whose absence I've found more and more conspicuous over time is interactive-compact. Given a current context, and impending context overflow, I know the directions that my mind is heading, and where I expect the development flow should be focused on.
But naive-compact is forced to just sort of guess at what is and isn't relevant from the prior work.
The harnesses have gotten better at some JIT ui stuff, throwing interview questions / forms at users. Compact is the ideal time for this:
Where are we headed here? (2-5 viable options, sourced from current context and imagination)
Then potential follow-up questions as required, but honestly I expect the single guiding answer there to improve post-compact performance pretty dramatically!
Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic.
He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. Please at least consider the possibility.
reply