Software is easy until it isn’t - the moment you need serious infra you’re in deep trouble and looking for an experienced partner or a saas solution anyway.
The days of b2b saas subscriptions doing one simple thing well for 3 or so users are however numbered.
I mean I understand that this can happen, my point is that we should code things in a way where the huge majority your code will still work no matter if you have an old rhel box or the latest archlinux, just like today we can port doom to pretty much any platform without too much trouble.
It’s very amusing to me that rust is trying to achieve a safety critical certification. Whatever version of the compiler that gets the cert will be cemented for the next 20 years.
Safety critical work / security-sensitive work is completely orthogonal to the rust release cycle.
Not to mention the vast amount of professionals who don’t have root on their runners / build nodes. Putting a ticket in every 3-6 weeks for a new compiler version takes… 3-6 weeks. People give up after a while.
> Not to mention the vast amount of professionals who don’t have root on their runners / build nodes. Putting a ticket in every 3-6 weeks for a new compiler version takes… 3-6 weeks. People give up after a while.
When I talked with the people at AWS responsible for updating both the myriad Java compilers in use as well as Rust for every project in the company, they claimed that updating Rust was always painless and didn't even appear as a blip in their radars compared to other similarly wide reaching updates.
There are governments plural which need to agree to stop and historically I can think of freon, leaded gasoline and IAEA and the latter didn't do as good of a job as it was meant to do (but arguably as good of a job as was politically possible)
Also arms control agreements, which reduced the world's nuclear stockpiles by 90%. In principle we could do something similar for GPU farms, since they're not exactly easy to hide.
Indeed. Even supranational coordination should be possible, given that extinction means extinction for everyone, and your excellent examples show the possibility. But then, here we are, and the FT just reported:
> Donald Trump rejects calls from tech bosses for AI slowdown
> President denounces demands for regulation as existential fears over technology move to the centre of US politics
You can order a custom crispred virus for what? $20k? $50k? Or make one yourself, if you know how, which you don't.
...but LLMs do know how; they can design and order one, or tell you how to build a lab to make one. If you ignore this option, you are simply lacking imagination.
Gasoline and lighters are abundant. It takes only slight steering mistake to drive into the crowd. Yet we are still here.
Yes, terrible stuff happens, but they are extreme outliers. Not sure how this works, but we can trust strangers to quite a high degree.
I have no experience in the matter, but surely designing viruses is not a simple affair. (In any profession, having a blueprint is not the same as having knowledge, equipment and practical experience; if somebody would try, my bet would be that they would die during early stages of the process, due to inexperience of handling hazardous materials; kind of like Mr Darwin looks after us)
Surely it isn’t, solving millennium problems on a whim isn’t, either; actually, it’s probably much harder, which is the concern. How hard is it to maneuver an aligned LLM to splice some smallpox into a generic research virus? BSL4 labs do things like this by hand all the time, so it can’t be too difficult and it only takes one lucky AI cultist for the rest of us to find out. I hope I’m wrong, but hope it’s not a process.
Math and computers have benefit of not needing materials. Add a button here, add lema and assumption there - all are almost zero cost&risk. So iteration/experimentation with llm in hand is possible to some degree even for laymen.
I assume virus creation is more like building a house.
It involves physical work and chemical reactions that need time to finish in correct order.
Each attempt costs a LOT more of time&money and involves actual risks.
All you need to pull that off is public info, student level knowledge and access to a lab with standard equipment. As you said labs do things like that all the time. Its already easy even without powerful AI.
The reason we dont see any serious attempts to create a deadly pandemic is not that we keep insane people out of labs, because there is no way to find out if someone is a cultist or murderous psycho when they have every reason to actively hide it. Its because a) most people are not evil and b) a virus that can kill everyone it touches would probably kill its creator first.
As someone with basically zero bio experience I think I could get a working smallpox or ebola level virus in a year or two if I had time to learn, lab equipment and Annas Library. No LLMs needed. But I dont want to do that, and I think its the same for all the people who could do it right now.
The idea that a few hundred apes with nothing but a bunch of rocks could one day land on the moon and come back to earth safely must’ve sounded ridiculous a hundred thousand years ago
It's not honest in AGI timelines (only biological ones). It just accidentally supports your unsubstantiated belief. Your belief isn't magically true because you're somehow able to see the future when others can't. You're just arrogant.
It was ridiculous, it took 100,000 years. If you built a recursively analyzing and improving structure out of LLM bits and it took 100,000 years to get to the moon, somebody saying that they were useless would have been right.
Call me when LLMs can get simple things right. Math is just the manipulation of symbols within established frameworks, we should be getting new math out of LLMs daily and we're somehow still not. They can't even do customer service, which is usually handled by 90 IQ people. I'm not impressed that they can find bugs; memory bugs are obvious when they're pointed out to you, and LLMs are entirely made up of examples and the relationships between them.
These companies are about to crash, and they're afraid they haven't reached the point where they'll have to be bailed out. I'm also subscribing to the conspiracy theory that the companies want the government to step in and create AI regulation boards entirely staffed by people at the current US frontier labs, so they can collude to both raise prices, to get government contracts, to make open/Chinese AI illegal, and to make things that were once easy to do without an AI intermediary impossible to do without an AI intermediary. Raising prices and forced purchases are the goal. They're trying to avoid having to compete, because as a business they're garbage.
Matt Stoller characterized their relentless press releasing as something like "my dick is so big that it has to be regulated." It's such an oversell for something that is not showing up as productivity gains, and anybody who has personal experience with knows is incapable of doing more than three things correctly in a row.
‘Figure out fusion powered CO2 sequestration’ is much better.
reply