This is an incomplete and quite superficial view of what is going on out there, in my opinion. I've worked on plenty of projects where the assumption was that since the round-trip to the server is going to take almost 100ms that'll dwarf anything that's going to happen on the server itself, justifying poor choices that lead to potentially adding a whopping 100ms onto that number. These numbers only get larger with a larger perceived "Nothing we can do about it" budget as well, programmers often feel justified in doing just about anything once round-trip time grows, not understanding that they're just adding to an already existing problem.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.
I don't think it's superficial, but two problems adding up. Previous poster is talking about general latency issues because everything is networked and potentially quite far away.
What you point out is slowness once you hit the entry point. Go, or similar languages, as a server language platform could have solved that problem from a computational perspective. But it did not for the most part. In my opinion people choose the faster stuff because it's cool and they have more wiggle room to cram in to get back to the slow status quo.
Everything is overengineered, software or distributed architectures, sound to naive human logic but alien to computers. It's an cultural problem, development is so deeply entrenched into "business logic" that the minimal viable and computational economic solution isn't even on the table. I don't even think it has to do with cost or feasibility, it's just that your random e-com manager wouldn't know what to do with you, if a programmer really starts talking about hardcode tech stuff.
A friend of mine was once tasked with writing a kind of simulation that simulates millions of scenarios per session/run, and searches for a best-so-far solution while doing so. He proposed writing it in Rust (justifying it as: fast, low level, fewer memory bugs, fewer parallelism bugs (so potentially faster than "fast")), and management over-ruled them, and insisted on using raw/plain Python (without even an underlying C library), "because that is the industry standard", and "premature optimization is the root of all evil", and "nobody else knows Rust"[0].
Another friend, worked at a company, that got a new manager (I think as a result of a merger), and that manager halted all work on "yak shaving" projects. These "yak shaving" projects were things like logging, and debugging, and some kind of integrity-verification. When asked why they were being halted, the new manager said: "none of our customers asked for any of these things". When told that these things enable the team to produce a better product for the customers, the manager (I am told) looked at them with confusion and suspicion. Those projects were never improved since, and the product stopped improving as well. I am not sure if it affected their business (the pandemic was much more distortive).
What you call "business logic", is not even logic, and it has little to do with business. It is what Feynman called a "cargo cult". The obvious name for it is "cargo cult business management/logic".
It truly is embarrassing and shameful that after decades of idiotic decisions, it took a _trillion_[1] dollars of investment into a chat-bot technology, to finally crack open _one_[2] door to slightly less idiotic decisions, while opening dozens of new doors to decisions of an unknowable character.
Most companies (and, consequently, their engineering organizations) are simply _cosplaying_ as the things they are supposed to be.
I do not see how an AI assistant (or any kind of assistant or consultant) can save these fools from themselves. The only logical explanation is that most software companies are cursed -- you would have much better luck engaging a witch-doctor.
[0]: Nobody else knew C or C++ either. The fact is, that nobody cared. In fact, even Go would have been a better choice than raw Python, but nobody cared. Even Common Lisp (which is at least as abstract as Python, and has native execution speeds (GC and runtime type-checking can be turned off for compute-heavy workloads that mutate data in-place)), is a better choice, and yet, an _abundance_ of obviously superior options (all implemented and maintained by obviously superior engineers) was not enough to prevent the organization from choosing an inferior one, and using it stupidly (without a fast native-code component).
[1]: I see estimates from hundreds of billions to a trillion, depending on how you count it.
On top of that: Creating software that isn't outright wasteful in terms of performance isn't even hard, it's just a matter of not doing ridiculously dumb things. The problem I've observed in teams I've worked with is that the majority of programmers don't even know what the dumb things are, and wouldn't know how to even approach making something that's halfway fast.
Edit:
Unfortunately I think posts like these are only going to make the problem worse, because now people are going to ask for voodoo solutions to performance issues, when the answer to their problems was usually just "Maybe stop creating wasteful intermediate structures and just walk an array like a sane person" in 99% of cases. The first leg of any optimization journey in the average programmer's code will likely net tens or hundreds of times faster code, and that's actually all people were asking for.
The knowledge required to make those changes and understand them is fairly minimal, but the kinds of people who have to create spinners for webmail interfaces, have their application add 150ms on top of whatever round-trip you have for processing things counted in 5 digits, etc., have never bothered to even learn those things.