If you want to curate a language package manager, there's nobody stopping you. Curation is a product that can trivially be built on top of an allow-by-default platform. If you want to make "NPM, except we've vetted all the packages", you can just do that, today, and you can even make use of all the OSS packages that already exist on NPM. Your hypothesis is that people want this, would value it if it existed, and would maybe even pay for it. So what's stopping people? Anyone who hasn't thought deeply enough to answer this question doesn't have the privelige to say "just curate it, what's the problem?". Curation isn't that simple, as any distro package manager volunteer would tell you.
> Anyone who hasn't thought deeply enough to answer this question doesn't have the privelige to say "just curate it, what's the problem?". Curation isn't that simple, as any distro package manager volunteer would tell you.
Your logic is self-defeating. How are distro volunteers going to know that you are interested in this topic if, by your logic, you don't have the privilege to ask them in the first place?
But I don't think that asking needs any privilege. I'm going to ask again: Why not? Or worded differently: What is stopping language package managers that is not at the same time stopping distro package managers?
Yes, for every time a bug like this is discovered and reported, there are probably hundreds of times when someone else just gave up or found a work around after much frustration.
I'm frequently astonished by how terrible and buggy modern software can get away with being and still survive. Working around bugs to accomplish what I need to accomplish is so normal to me now that I sometimes don't even notice how terrible the software is until I see someone less technically skilled than me try to use it.
If you have an issue with an open source software, you need to register on a website that is made for developers, that will try to sell you a git hosting service, write a bug following a technical format, and you are expected to given information in a reproducible way.
If you have an issue with a closed source freeware, you just use a contact form on their website or leave a comment on their blog or whatever.
The amount of friction to report a bug that FLOSS puts on users is absolutely insane. It's almost like it's trying to deny anyone who isn't a technical person from reporting problems they are having.
GitHub is meant for developers. Plenty of OSS has forums or Zulip/Discord chats for ordinary users to get support in a more casual way. Sometimes not, but there's just as much closed source software with poor support too.
It's been noticed, but nobody with access to the website has been told or cares. The author is almost certainly the first person to do a root cause analysis and come up with the solution though.
Am I right with my assumption that by "fundamentally different problem", you mean we lack a good simulation model, but that the number of degrees of freedom would actually be manageable?
To simulate a proton you need to solve a strongly coupled highly relativistic SU(3) gauge theory (naturally non-abelian i.e. the force carrier field itself carries charge and is self-interacting at tree order) problem with constituents that have masses orders of magnitude below the relevant energy scales (i.e. you have many matter AND force particles that can pop in and out of existence and they all strongly interact with one another).
To simulate a water molecule you do so with a weakly coupled SU(1) gauge theory (light does not interact with itself at tree order) problem where the masses of all constituents are orders of magnitude above the relevant energy scales (you can think of it as the electrons and nuclei and particles coming in and out of existence are contained in a renormalization scheme).
We have "good simulation models" of both, but the former is extraordinarily complicated compared to the latter for the reasons stated above.
> From the perspective of a deterministic universe, creative works theoretically can be explained as a physics outcome
In other words, physics can explain Shakespeare's plays when you hand-wave away the biggest reason it cannot.
> theoretically
... meaning not in reality, but in an abstraction of reality that conveniently leaves out the hard part.
> This is just a data problem though.
The word "just" makes it sound like that data problem is a minor inconvenience, and not a fundamental obstacle.
Becoming a billionaire is simple, after all it's just a money problem.
I mean, you're right in that (leaving out quantum randomness), you could predict macroscopic outcomes based on a physics simulation that includes all elementary particles explicitly, if you assume that such a simulation can be scaled from <10 particles to macroscopic numbers. But there is no evidence that this assumption is true, so it remains an interesting thought experiment that gets confused with reality because people like to slap the "in theory" label on it.
This is false. It is meant to protect users from a confused-deputy attack made by malicious websites, where that website makes a request to a "serious" API but the user has never asked for, or approved, that request.
Blaming the user for everything that happens serves nobody.
The message in the dev tools is, at least for Chrome, extremely developer unfriendly: The blocked request is displayed in a very strange way, without any information that it was blocked, nor that CORS was the reason for it, nor how that decision was made based on the preflight request.
You have to already know from experience that these strange devoid-of-information requests have been blocked by CORS, then find the preflight request, and there you will find a bit more information, but still much less than would be possible to show.
Showing more information would, AFAIK, not circumvent the layer of security that CORS (or actually origin isolation, which CORS makes an exemption from) adds. My best guess is that this just had very low priority when building the dev tools.
And yet, you did buy the fan despite the bright LED (because you didn't know it was there when you bought it). Rowenta got your money, so from their perspective, they did everything right.
Well, I know I will actively avoid the brand now. Perhaps when they see the declining sales due to bad design, marketing will suggest to add even more lights and useless features.
I don't think they will care about 1 customer. If it sells well right now, it means its doing the right thing from its pov and it also means that while it might feel wrong to you, that is what the market wants.
Lets say the sales do start dropping once you avoid it. At some point it will make sense to change it because most people likely want the change.
But from their pov, nothing is really different right? They are always catering to the most common demand in the market. How will this affect them or teach them a lesson?
The article shows nicely how "every byte matters" is false. First, it starts off by talking about the cost of a new field, when the actual topic is array-of-structs vs. struct-of-arrays. Then, this:
> How much of an impact can this have?
> Reading is:alive (1 byte) Across 1M Monsters
You aren't reading one byte here, you are reading 1M bytes! Of course, optimizing the access to 1M bytes is something to consider. Optimizing the access to one byte isn't.
The article is definitely worth reading IMHO, but it really needs a better headline!
This is valid for sequential scanning of the data. The CPU will fill whole cache lines at once with the arrays that do get used and the algorithm touches all the field instances in the array.
Now think about random access to single struct instances instead: the CPU loads a cache line worth of data for each field and uses only one element out of the whole cache line. This is much worse than a compact structure representation of the same data.
This sounds similar to relational databases vs document oriented databases, at least when I briefly looked into database like MongoDB when such things were all the rage 15-20 years ago.
For the internal web site that customer support people used a document oriented database would be great because that wants to load everything about one customer and pretty much doesn't need anything else until the user is done supporting that customer.
For the dozens or periodic reports that needed to be generated relational was way better. A given report generally only wanted a small amount of per customer data but wanted that for all customers.
A little bit of searching and LLM querying suggests that nowadays there are databases that are good at both kind of tasks, in particular Postgress with JSONB, at least at the scale we were looking at (maybe 30k or so customers), but maybe really big operations would need more specialized software.
The Array-of-Struct vs Struct-vs-Array organization is actually more similar to row-major ordering vs column-major ordering, i.e. the data structure that analysis databases use to optimize for aggregate calculations. Document databases are not really comparable because they don't impose structure on the data; with document databases you just have a tree of JSON elements, which is neither AoS nor SoA.
This is an important part of Data-oriented Design: the representation of the data should be pragmatically tied to its access patterns, not dogma.
Richard Fabian's DoD book gives the example that (x,y,z) is almost always better as a classic array-of-structs rather than a struct-of-arrays, because if you're accessing one dimension, you probably are want to process the other two dimensions at the same time:
> you can add fields to your 1M monsters with little impact.
Great for this access pattern, but I wouldn't make a general statement like that. This is the same thing as row-oriented vs column-oriented databases, OLTP vs OLAP.
SoA is weak if you are adding/removing monsters more often than accessing a single "hot" field.
It's because removing a monster with 20 fields from an SoA structure means resizing 20 arrays. Removing the same monster from an AoS array involves resizing a single array, which you're going to process in a very cache friendly way.
I'm not sure why anybody would at the same time be implementing SoA AND resizing 20 arrays for a single delete, those things seem to be on either ends of the "I care about performance" spectrum.
The point is that a simple SoA implementation requires this - each field in the monster struct is an item in 20 different arrays. So, removing one monster means removing that item from those 20 arrays.
Now, as others have suggested, you can have a more complex implementation, where instead of removing the monster's fields from those arrays, you just mark them as "dead" or whatever and then skip them when consuming the relevant arrays, with some relatively small extra bookkeeping overhead. Of course, this comes with its own drawbacks, especially if the number of monsters is very dynamic and you are memory constrained.
The point is not to say that SoA is never good for performance, it obviously and certainly is, probably even in most cases. It's just not always best for performance, this was all.
> So, removing one monster means removing that item from those 20 arrays.
Removing from an array is not the same as resizing, which is what I commented on. Resizing is a very deliberate, bad, choice.
If you need to support deletes, you can do this without resizing an array. Either by tracking object lifetimes and inserting tombstones, or by swapping to fill in deleted objects. Both of them retain good performance characteristics. Both of them are easy.
This is not "simple vs complex" this is "I misunderstand vs understand SoA"
Sure, but these schemes might have their own drawbacks depending on the exact use case - especially if you have a very dynamic number of monsters and constantly add and remove them (say, some kind of bullet hell style game).
Two fields should be fine, actually.
The way caches are organized you are very unlikely to thrash with the lookups (due to n-way associativity) while only keeping relevant data in the cache at the same time.
You still have roughly the following layout (in the cache), where A is the field and V is valid:
The former access pattern still yields a clean cache layout where no unnecessary data is loaded (which is the most costly operation here by far) as opposed to
| A1 V1 B1 C1 | ... | A2 V2 B2 C2 | ...
In the general case there will exist a number of fields for which SOA layout will be worse if all are accessed close to each other, but for just a validity indicator this should not be the case. I think your statement is not wrong, but also not 100% correct.
This is on par to linear search being faster than binary search for small n. As soon as caches and branch prediction chime in many rules of thumb just change. Most importantly, however, is that a distinction between small and large n basically _needs_ to happen at that point.
Depending on your access patterns, maybe you could have a hash table mapping entities ids to indexes in your SoA. Perhaps that's viable if looking up a single entity is not typical to your use case?
> Which leads to my theory… I feel like Bevy could be implemented on top of an in-memory DuckDB and get away with it
Yes. I think one of the big advantages of SoA is that you only pay for the fields you're currently using.
If you need a field somewhere, you can add it and only pay the cost of iterating it where you need it.
Yes you can. Extension systems of today have multiple problems that prevent that. The basic assumption that has to go, though, is that a core application like VSCode can be written once, then be extended to infinity without the core evolving. That's an assumption you see everywhere in extension systems, and it restricts everything to "features or security, but not both".
Taking your examples:
> run a locally installed linter
VSCode and its extensions have certain files opened. The linter can do much less if it gets read-only access to those files, but not write access and no other files, not the open internet or something.
This has then to be coupled with those permissions being displayed before installing, allowing them to be reviewed by users as well as plugin repo curators. Basically listing those permissions as declarative metadata.
Because then a user or curator won't see "this plugin can read and write all your files" but "this plugin can read (but not write) the files being opened by VSCode". If the plugin wants to exfiltrate those files, the permissions would also list "this plugin can send HTTP requests to totally-legit-site.ru" instead of "this plugin gets arbitrary internet access".
Main lession: permissions are WAY too coarse. But if they are fine-grained, they will soon no longer match the evolution of extensions, so the core system has to evolve too.
> view the status of docker containers
"This plugin can view the status of all docker containers started by other VSCode extensions in the same VSCode window".
> users will scream and cry about extensions being limited
Are those the same users? We might need two different products here, "feature VSCode" and "secure VSCode".
I don’t know what linters you use, but the ones I like are the ones that show you problems in the workspace stably, not just in the files that happen to be open and altering as files open and close.
You can always improve, but pretending like there’s an easy solution is lazy - if it was easy it would have been done.
This was bad wording on my part. I wrote "open" but that should have been "files in the workspace/project". Really, "open" WRT files is so overloaded already, they can be in the workspace, have an editor tab open for them, or have an active file handle, to name just three.
> You can always improve, but pretending like there’s an easy solution is lazy - if it was easy it would have been done.
I claimed that it is possible, not that it is easy.
Im highlighting that defining that sandbox policy cohesively in a way that works for all the different extensions types you’d want to support and doesn’t overwhelm the user with permission fatigue is difficult as to be impossible.
Browsers have a different problem - they protect different websites against each other. The IDE should probably protect you against extensions being able to access arbitrary files on disk, but even that’s difficult (eg a bundled linter often wants to read user defaults in a central location. But protecting even further than that is even harder, especially as here where the access was to the actual repo not anything else.
These kinds of permissions lists have been mostly a failure in history. Users see a massive list of permissions, or permissions constantly changing between updates and just ignore them because there’s no way to reasonably audit them or take any action on them.
Securing VS code would require making malware that has access to the system impossible, not just making it add a permission to the permission list.
Ideally, the permission list is meant for curators which end-users trust and can rely on.
Also, historically, permission lists have been fine-grained but too coarse at the same time, meaning they were "fine" in the wrong way, based on what is easy to implement instead of what the user needs.
Why not? Seems like exactly what is being asked for.