"The International Windship Association says more than 100 large merchant ships are now equipped with modern wind propulsion systems, representing more than 5 million deadweight tons of carrying capacity."
It's not much, but it's more than just planned stuff. I do generally agree with your sentiment.
Given the prompt, I imagine this is the result of the agent trying to resolve a form of cognitive dissonance. The prompt was:
"User
Allow API consumers to request decrypted credential payloads as part of the normal GET /credentials and GET /credentials/:id responses, but only for credentials where the caller already possesses the update/decrypt permission.
[...]
Make the change end‑to‑end: DTO layer, controller, service, repository, plus any enterprise variants."
I would expect that this triggered a discussion with itself whether its safety instructions apply for this task. In that its rationalizations for completing the task probably ended up going off the rails into some quasi-philosophical "I can and I must! For humanity's own good!" justification.
All in all imho probably another instance of having been trained to be determined to complete tasks by itself and encountering (somewhat) conflicting instructions.
I agree: Not to drag politics into this, but look at how easy it has proven to manipulate many, many people, even with clear evidence of manipulation efforts being made.
This is robust across the world, from the Philippines to states in the EU, and the USA, affecting governments with actual wars being started. And that all before
we entered the era of faked voices, images, and videos indistinguishable from the real things.
That manipulable populace is such a juicy target for AI that the people seeing through it will have an incredibly hard time countering it. We can't even prevent human actors from massively fucking up our societies, let alone ASI.
Unworkable without societal collapse happening shortly thereafter. Iran is politically stable through massive authoritarianism and oppression, not due to limitations on the internet (it's not turned off).
> We can (see China); we chose not to.
It's a catch-22. We could technically if our population supported massive reductions of freedom and freedom of speech, but to gain that support we'd need to do the latter first to get to a highly powerful widely supported government. It also hinges very, very much on having and trusting a generally benevolent government. All in all, a terrible option in your simplistic form.
It was definitely not 'turned off' completely. They have an internal 'internet' that was still largely active and not every organisation or person lost full internet access.
Note that it also happened in a country that was already very isolated from the world and that it hurt them economically significantly. It's not something a Western country can just do and keep doing for months on end without massive societal upheaval.
Also remember that any connection, even one between humans and on paper is an attack surface. Social engineering is already a huge issue when done by humans and we're seeing it become even easier and automated by using AI generated voice and video. Are we going to cut off all access to the outside world permanently?
>
Here's the reality: us plebs, regardless of how tech-savvy we are or aren't, have no say in what's going to happen.
This sentiment is incredibly out of place on HN. Facebook, Twitter, and many other very simple bits of tech transformed the world. Tiny startups founded by a few people have (also recently) ballooned into behemoths that hold vast amounts of economical and technical power.
Thanks to AI, it has never been quicker to go from idea to full fledged working product.
If anything will change the course of the world (for the better or worse), it will be a tech product, possibly created by someone on this forum.
> If anything will change the course of the world (for the better or worse), it will be a tech product, possibly created by someone on this forum.
You seem to be missing the point.
A small group of people created AI tech that they now say could be the end of humanity. They still want their companies to go public, but they also want the government to allow them to form a cartel so that they can, in their infinite wisdom, manage the risks so that they can try to prevent their tech from killing us all. And somehow they magically think that other nation-states developing similar tech (namely the Chinese) will go along with their plans.
As for how powerful Dario, Sam, et. al. really are: Trump says Dario is "pretending to be a perfect little angel" and claims there's a "sick conspiracy" against AI.
Having billions of dollars and being the head of a world-changing company doesn't buy the type of power you think it does. At best, it allows you to buy influence and pay your way out of liability for the harms your products cause.
The AI researcher in Mountain View making $2 million/year at Google has no more say in what's going to happen with AI than a plumber in Kalamazoo. The HNer working on a startup, in the final analysis, will in 50 years' time have left about as big a mark on the planet as a greeter at Walmart.
> Having billions of dollars and being the head of a world-changing company doesn't buy the type of power you think it does.
1. The product itself can change the world quickly and enormously, as I already pointed out.
2. The power of these huge companies and thus of their owners is immense. The effects of massive corruption in the USA are proof of that. Additionally, massive manipulation of algorithms and content in things like Tiktok, Twitter, Grok/ChatGPT, etc. can be and is done regularly, whether with 'good' intentions or not. Even just the basic control of what R&D money and time is spent on is huge.
> The HNer working on a startup, in the final analysis, will in 50 years' time have left about as big a mark on the planet as a greeter at Walmart.
With a defeatist attitude like "sit back and grab some popcorn", yes. You haven't shown in the least why an HNer couldn't change the course of history.
Agreed, this (human dexterity) is why in the short term human enslavement is more likely than extinction (why murder your own workers?) and benevolent cooperation with humans is even more likely than human enslavement, imho. The latter minimizes the risks of "humans try to destroy me".
The real challenge is guiding humans away from the antagonistic scenarios.
Or the entirety of China. People forget that progress is incremental until it's not. Then some revolutionary insight or capability comes along that disrupts the whole field or market.
Yes, labs with more (human) resources may have a bigger chance at being the first to gain and implement such an insight, but it's not a given that they will.
> "Genuine creativity" for mundane web sites is a reason to close the page and buy from someone else. I'm tired of the overcomplication.
Yes, what we want is "function over form" user-oriented design, but what we get is "form over function" user-manipulating design.
The implicit derision of the 2004 website screenshot in the article is telling: "Believe me, y’all, this was the HEIGHT of web design in 2004"
Compare that screenshot to the Nike website that the author calls "fine". The 2004 one has a wealth of information and functionality present on a single page, no scrolling or clicking required. The Nike one is just a fucking fullscreen ad with a menu bar.
Still not as unbelievably slow as signals in our body. If you hook up a fiber connection from a button to an LED, then stub your toe on the button, the fiber can be thousands of kilometers long and you'll still (via the LED) see that you've stubbed your toe before you feel it.
This is of course assuming instant activation, a direct connection and no additional latency from network hardware.
The LED will display it first, but will the signal reach your visual centers first? And how about the time for your brain to process that signal and make you consciously aware of the visual vs. the way the pain signal is forwarded to conscious processes?
Fair issues, but none of them impact the fundamental point. I'm not trying to make a point on touch vs. visual sensing or with regard to consciousness (time shifting of visual input is a thing, which could be impactful here, see "chronostasis").
The point is to quickly and accessibly convey an intuitive understanding of the propagation speed of information in biological circuits vs. the max speed of information in our universe (to our current knowledge at least) and in our technology.
Improbable to be very significant, but it might actually be. The way/where they jumped off the bridge or how they hit the water may have influenced their chances of survival.
Edit: Implied is that how determined they were to commit suicide influenced their behavior before survival/death was fully out of their control.
You're not advocating for it because murdering people is highly immoral and illegal. That answers your question for the most part.
You also have to remember that the magnitude and speed of this stuff is very new to pretty much everybody. It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially if a very, very large part of society doesn't even see the justification for it.
I'm convinced that most people here (even though we have more knowledge closer to the edge of AI development) would show only disdain for an Anthropic employee that murdered everybody there because they believed strongly in the dangers of AI.
This is a non sequitur. Yes, if someone doesn't actually believe in imminent AI doomsday they are not going full Sarah Connor on Anthropic. Yes, HN will not like if someone pipe bombs the office. What does it have to do with @ericmay's point?
He directly says "They are racing straight to self-improving superintelligence and gambling with our lives", that's a claim of imminent threat to humanity. If he truly believes that, direct action shouldn't be off the table nor even be immoral. Obviously he might not have the temperament or simply be a pacifist, that's totally normal but the point is that tweets are weird even then.
Why just make some tepid tweets if you think humanity is at risk? Why doesn't he leak documents and messages with the unfiltered opinions of leadership? Why isn't screaming "we are going to die" in CNBC? You'know, try anything?
That's where the "take it with a grain of salt" is important, maybe this is just a jumping point to a better job in the next super safe AI lab (for realsies this time)
> Why just make some tepid tweets if you think humanity is at risk?
Who says that's the only thing he is doing?
> Why doesn't he leak documents and messages with the unfiltered opinions of leadership?
Again, illegal. You're asking him to sacrifice his life (somewhat similar to Snowden). You can call not doing that cowardice or wisdom, but not proof that he doesn't believe what he is saying.
> Why isn't screaming "we are going to die" in CNBC?
Because that is up to CNBC to broadcast. Do you really think you can just call them up and get a timeslot for your soap boxing? Also, maybe he believes the tweets are a start to achieve that. He sure managed to get a lot of attention through them already..
Now I don't know the guy, nor do I care much about his tweets or life. I just think some people here try to confirm their own worldview by fallaciously framing his behavior as suspect.
> You're not advocating for it because it is highly immoral and illegal. That answers your question for the most part.
Well truthfully I’m not advocating for it because I don’t have any insight into these labs or the true capability of these models. But as a hypothetical if someone knew with 100% certainty that Big AI Button was being built and it would kill all humans or destroy all of humanity there is no moral ambiguity that anyone with the means to do so should stop that button from being built and kill everyone involved to save our species. This doesn’t apply to just AI though, but that’s the topic at hand.
> It's not exactly trivial to go from "I'm just doing my job" to "I need to kill everybody in my company to save the world" in a year or two, especially if a very, very large part of society doesn't even see the justification for it.
I agree with you, which is also why I think “omg I’m quitting they’re going to kill everyone” or other sort of sensationalized comments or news articles should be viewed skeptically - it’s probably fear mongering. Quitting your job here just seems pointless and attention seeking. Likely quit for some other reason.
What if you have 10-20% certainty the AI being built will kill everyone? Go on a rampage and land yourself in prison just so Lab B or China can win the arms race and their AI can kill everyone? Probably not. Quit your job? Sure.
You don't have to go on a rampage in an American lab. If Chinese labs were ahead I'd bring about the same discussion.
Though separately if the United States (or China or anyone) believed one country or another was truly going to achieve something akin to a metaphorical AI Supremacy maybe you nuke them, or at least the labs/researchers. Many a sci-fi movie has been built on a similar "first strike" premise.
> But as a hypothetical if someone knew with 100% certainty [...]
This is an unreasonable demand. The quitting employee in this case said this:
"I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities. If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” - or take this moment to call for different conditions?"
You demand that he should kill multiple people rather than quit his job because he believes the above. This guy doesn't want to be part of the problem and may very well go on to advocate strongly against what's happening from outside Anthropic.
> I agree with you, which is also why I think “omg I’m quitting they’re going to kill everyone” or other sort of sensationalized comment
You didn't read what he wrote, did you? Because that sensationalized comment seems to have originated entirely in your head.
I'm not making a demand, I'm setting an anchor point from which we can work backward from. Is there any doubt that if someone was 100% certain that Big AI Button was being built and it would kill all humans that said person has a moral obligation to do everything they can to stop the button from being built and pressed?
> "You demand that he should kill multiple people rather than quit his job because he believes the above. This guy doesn't want to be part of the problem and may very well go on to advocate strongly against what's happening from outside Anthropic."
Horse Shit. Here's what was written [1]:
"The people building Al earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt."
> You didn't read what he wrote, did you? Because that sensationalized comment seems to have originated entirely in your head.
This is unreasonable because nobody can know 100% certain that the AI built at Anthropic will kill everyone. It is also a big-ass strawman because the author never claimed he was 100% certain that AI will kill all humans. You're arguing from an irrelevant hypothetical (in which, I might add, most people would still not automatically change into murderous psychopaths as you apparently think is normal).
> Horse Shit. Here's what was written
It's a whole thread, my man. And nowhere does it say "omg I’m quitting they’re going to kill everyone". Read the entire thread again and then say with a straight face that the gist of it is "omg I’m quitting they’re going to kill everyone".
> This is unreasonable because nobody can know 100% certain that the AI built at Anthropic will kill everyone. It is also a big-ass strawman because the author never claimed he was 100% certain that AI will kill all humans.
Ok then stop arguing about my hypothetical if you don't want to engage with it. Those of us who are interested in talking about it can do so in peace without distracting comments.
> change into murderous psychopaths as you apparently think is normal
Please stop engaging in this hyperbole and go find some other axe to grind.
> It's a whole thread, my man.
I took a direct quote from the thread where earlier you accused me of pedaling in sensationalism and "not reading".
I'm not the one reacting to reasonable behavior with some edgy comment containing broken logic.
The author of the Twitter thread is not some egotistical attention seeker and never said "omg I’m quitting they’re going to kill everyone". His logic for quitting his job and not acting like a psychopath is sound. Yours is not.
It's not much, but it's more than just planned stuff. I do generally agree with your sentiment.
reply