Programming with AIs requires new architectural structures designed around the all-too-small context windows of the AIs themselves. Highly modular, highly semantic, highly traceable, good at deleting and refactoring/removing things they no longer need, good at identifying the core truths/principles/designs they need to uphold, good at recognizing what is mere interface from architecture from data.
This is the great orchestrator game. It might be the last game. I dont think it lands precisely in our previous "best practices" though, and I don't think it's trivial to solve for. I also don't think it's going to be a human-centric design. I think we're generally better off rethinking fundamentals and designing from the ground up on this one. I think it's also a lot of fun to start thinking in these new ways.
There will always be programming, just like there will always be oil painters and historians. But this is about to get industrialized into stable emerging factory forms, and there's a lot of ways to design those right now. And after that us artisans will either need to find new unfinished aspects of programming-like design or retire. At least take solace that the rest of the world will basically simultaneously be doing so too.
First, the "all too small context window" reminds me of GetRight, mp3pro and a lot of other artifacts we invented in the 80s and 90s to deal with the limited internet bandwidth of then. We created artifacts to "achieve things" with the limited technology at hand. In 2026, ive got a 500 mbps simetric home connection. The same way, RAGs and all related stuff are artifacts to help us deal with today's tech limitations. In 10 years, it will be a non issue.
Second, I agree on the architectural patterns. Building (commercial) software will become a real Engineering career. Engineers will define the blueprints of the functionality of software, as well as the "materials" (technology, practices to use, measurement limits, ranges, etc) of tge software to be built, the same way Architects and Civil Engineers do when designing a Skyscraper or a bridge to be built.
As you very well put it, there will still be the "artisan" programmer, who builds whatever code by hand. But that will be as a hobby or some personal project.
If you care about architecture and principles, the AI is excellent in architecting around that goal too and/or helping you articulate your own intuitions. You just need to be willing to sacrifice code delivery speed. It's far from an either/or.
I have found different, often even when doing something like coming up with a database schema, the solution it chooses is not the best and and can be even terrible sometimes. It could be I suck at prompting.
Yes but those are all semantic tasks which can be done and understood in time by the next iteration of models training at that meta-level of architectural analysis.
AI deeply understands what Bun, Zig and Rust are, how they work, can conceive of the before and after architecture (probably the hard step here), can conceive of the goal of the rewrite, and can verify (using Lean and machine-checked proofs of the before and after expected states) the final result.
They just needed us to ask. Build a sufficiently general research program that can find and iterate on ideas, and it will do the asking itself.
Sure, maybe step 2 can eventually be replaced by AI. But step 1? Why would the AI even conceive of the idea of doing a rewrite in the first place?
And rewriting is just an example. There are tons of architectural decisions that need to be made every day in the building of advanced software. This is no "correct" way to do it that the AI can know in advance, it's a technical decision to be made by an engineer.
Maybe it sees a bunch of segfaults in the Zig codebase and decides a memory safe language is better, just as the human did in Bun's rewrite. I'm not sure why you think it wouldn't have been able to conceive a rewrite.
Such an AI would be unusable in production because there is no bound on the work that can be created and external side effects that could happen. All projects have real world constraints and side effects, budget, customer requirements, etc. Imagine a non-technical person prompting the AI "make the app faster and fix all the bugs" and the AI autonomously decides to rewrite the whole production app and all its dependencies in Rust that is live and serving thousands of users. Doing a full rewrite would take days to execute, and have all sorts of side effects on the actual users during deployment (even if it was done 100% correctly, work has to be paused, affairs need to be coordinated with real customers, etc.). Even if the AI has that capability (and I do believe it is possible, even with today's models), that's not what a business owner wants. You don't want to use an AI which, if you gave it a 3 word prompt, it could suddenly decide on its own to rewrite your whole entire business. An AI that had that much autonomy could just decide on its own to pivot your whole startup and sell something else. A business owner wouldn't even want to use such an AI which could have such large unbounded side effects.
You are essentially describing an over-eager developer having to run decisions up the chain of command to senior devs / product owners / company execs. I assure you, every one of those roles is being trained on to balance their own wide breadth of concerns, and is unlikely to advise something like a live rewrite of a production app or an entire product pivot - unless it actually made sense from an economics/management perspective. Every step bounded by its role and responsibilities.
Turtles all the way up, and all the way down. AI getting this good at coding means it's a hop skip and a jump away from being this good at every white collar role.
Yes, but the entire point is, you can't have a reasonable chain of command or any kind of sensible decision process without human judgement that has engineering expertise, so no, engineers are not going to be entirely replaced.
The reason human judgement is necessary is not because it's "better". It's because the entire reason the business exists is to execute the human's judgement. There is no "right or wrong" way to build a startup if different paths are all profitable, but the human decides what they want to do.
Could Bun have survived and done well even if they didn't rewrite the codebase in Rust? Yes, it would've been fine. But the human decided they wanted Rust, that's a different path they chose with different tradeoffs. The AI has no skin in the game , no ownership, it doesn't belong to the AI. The AI only exists to serve the human. And if the human wants to make good decisions, inevitably they need engineering expertise (or an engineer) to weigh the decision.
I feel like leaving it to decide everything would not produce a good end product. It feels like just having it decide how GC should work, how the function call stack should work or any of these rather simple but actually requiring lots of decision and thinking how things fit together (and pick a good solution out of many seemingly good solutions that can come back to bite you), couple this with the tendency for AI agents to tend towards adding new code and building over features I don't think you end up with a good solution.
On formal verification having done it only in an academic sense and looked over at projects like seL4 and quite interested in that project. It feels like actually proving useful properties of programs for real programs even ones with well defined domains and easier to model such as interpreters or compilers it seems that it will just prove theorems about properties that hardly matter or don't even matter at all. See how bad it still tends to be when trying to get it to write tests.
I would be interested to know if their has been an agent that has actually utilized formal methods such as Lean or Coq or Isabella to prove properties of programs in an automated way as you suggest, I have only seen it proving mathematics and or searching for counter examples, not writing Curry-Howard style proofs.
Shoutout to Toph https://x.com/VictorTaelin who's been building a programming language out of formal method Lean proof checkers, which probably would indeed have the scope you're looking for.
I do think the current state of the AI just using rigorous tests and rendering inspection loops is still far more than most programmers did for the majority of apps, but yes I think rigorous formal verification will come too. Worst case it's gonna be something like a MechanicalTurk pipeline having us humans verify narrowed scopes the AI can't confidently inspect (yet. while also training on those results for next iteration)
As for the decision stack of what makes a good end product - turtles all the way up/down. The ProductOwnerAI role decides those things, and it will likely do so with the same deep skill that programming AIs are currently hitting our profession with. Optimizing enormous breadths of concerns and simulating results is AI's main specialty.
More precisely, a "perfect solution" doesn't exist. It's all tradeoffs given your goals. Someone needs to make the decision: 1. which tradeoffs are worth given your goals 2. which goals are worth defining or redefining. And do that effectively, you need to understand the problem, which goes back to engineering.
I hope that after a short period of delusional expectations and layoffs from employers we're at least left with a more consistently competent set of professionals in our industry. Some people have imposter syndrome. Others are actually just imposters.
> Some people have imposter syndrome. Others are actually just imposters.
I'm sorry to say, but AI coding assistants paved the way to professional imposters whose only skill is prompting a model to do something. I already had the displeasure of working with a software engineer who not only introduced a bunch of regressions that by mindlessly vibe-coding things against the requirements but also complained that not having credits to use the most expensive frontier models was, and I quote, "stifling my creativity".
Nonetheless, would still trust Claude to be generally more reliably competent in a random area of software engineering than the average professional. Sure they might still be better in a particular area of their expertise, but we've all had to play the imposter before on the stuff we care less about and figure it out as we go. AIs are still inferior sometimes but usually a decent 7/10+ on most topics - which really fills the gap.
I hope that after a short period of delusional expectations that top management is going to reap the rewards of these AI capabilities we get a whole lot more comfortable with just doing the full business stack - including management, sales, branding, etc - and hierarchical structures crumble into flat collectives of do-everything true-generalist programmers.
I think this is a very reasonable middle-of-the-road AI take and our likely future. Just with the caveat that there's still a major threshold being hit here where we jump up to a new casual capabilities class where it becomes silly not to use AI for the majority of work, but there are still some high-intricacy problems which become much more load bearing than they ever were before and our new abstraction level doubles down on those.
I would like to submit that the high-intricacy work congregates in Protocols themselves, and we start seeing the cycles of development and all the ways to direct AIs, programs, inter-person/inter-company interactions, etc etc all as types of protocol design - and studying those rules of interaction themselves becomes the new job of a programmer (systems architecture). What used to be hard rules and deterministic programs becomes soft self-governing tendencies and probabilistic behavior that can nonetheless be managed and bounded with the right system, but it's new and weird and more akin to management or herding cats than architecture. This is still very different from what most of us were working on before AI, but it's still familiar - especially to those who worked on internet protocols, or defensive UX design around users, physical engineering systems, or team management. Less programming languages, more - control theory, flows and throttles, quality control, design theory, etc. And clearly the field is still wide open as everyone seems to be experimenting with their own take on the AI orchestrator.
This. AI can, should, and will erode all legacy companies into intelligent utilities - with an end state of nearly-free open source utilities.
Anyone concerned with concentrations of power and abuse of AI should be focusing on getting open source work to keep pace with decentralizing that power into accessible free tools for the masses.
This is the great orchestrator game. It might be the last game. I dont think it lands precisely in our previous "best practices" though, and I don't think it's trivial to solve for. I also don't think it's going to be a human-centric design. I think we're generally better off rethinking fundamentals and designing from the ground up on this one. I think it's also a lot of fun to start thinking in these new ways.
There will always be programming, just like there will always be oil painters and historians. But this is about to get industrialized into stable emerging factory forms, and there's a lot of ways to design those right now. And after that us artisans will either need to find new unfinished aspects of programming-like design or retire. At least take solace that the rest of the world will basically simultaneously be doing so too.