You make some great points that I largely agree with. I'm wondering whether I'm misrepresenting you though, because I'm not arriving at the same "everything looks like a nail" conclusion.

One thing I'd like to surface is the distinction you're making between software and physics. You point out that social and medical sciences are only now becoming approachable with scientific methodology. We can now, to a higher degree than before at least, model and predict what'll happen in social and medical scenarios. Isn't that an excellent example of how subject matters can be virtualized? I can't find good arguments for why this would stop there; why wouldn't our scientific models improve to such a degree that we can predict and "interfacte" with more and more of reality. Put plainly: why can't experiments be done in code one day? Particularly if we imagine having better robotics.

Do you see my point about closing the gap between the theorizing and experimenting? Or are they fundamentally different things? The "reasonably skilled person" seems like an excellent heuristic > 2 years ago that's rapidly being replaced by automated thinking machines.

It's certainly possible to automate experiments. In fact my job is developing equipment for this purpose, and all of my own experiments are controlled by code. Most measurement instruments have run on code for decades. You don't manually turn the knob on a Wheatstone bridge to measure a voltage any more. A yellow handheld device with a LCD measures it with greater accuracy, or a probe that plugs into a computer. The code living in an experiment like the Large Hadron Collider is extreme.

The distinction between theory and experiment is somewhat arbitrary. Historically, scientists were better at one than the other, with a few rare people being known for both. To make an analogy, there's also no immovable boundary between hardware and software, since most hardware runs on code, but these tend to be different jobs and even departments.

What tends to happen in science is that when a subject is virtualized, e.g., when it can be modeled by a satisfactory theory, scientists move on to new frontiers. Many of the most interesting areas of work are where both theory and experiment can challenge one another.

I've had managers and even engineers ask me in all sincerity why we still need to make measurements when we can just model the data.

I've seen quite a few academics wrangling with patents. What should come first? Proliferation of the sciences or (potential) profit? Your point about the golden goose is pretty interesting and others in this thread has pointed to the misaligned incentive structure in academia. I'm curious, do you think things like JOSS could help open up data and "the golden goose"? I'm not sure the funding bodies I'm interacting with would respect this kind of initiative, but maybe it's just a matter of time.

My experience in academia was that data management is not well respected. Universities often see data as part of the library system which leads to a disconnect between data and research.

I’d suggest data management needs to be a well paid career in academia, with senior level influence to enable long term open access to data.

I was on a grant from the Wellcome Trust putting an epidemiology database online. This remained available until the PI moved on and their successor was decidedly less public spirited, so sadly it’s not available anymore. What can I do about that when I’m seen as a technician? I have over 8000 citations from my database work but there’s really no career in academia for me.

In principle I would agree. But, on a more philosophical level, couldn't you make that same argument about C? Or even assembly? Or even digital computers? Less facetious, it seems to me that the particular abstraction is less important. As long as it's unambiguous and widespread.

I think there is a preferable category of abstractions that are adequately unambiguous, hardware-agnostic, and only occupy a thin layer of the stack. The argument could sort of be made for C, but not native assembly or digital computers because that would push hardware towards centralization.

A useful way to think about this is to consider existing human languages like English or mathematical notation. When you read these English words, it's like you're executing a program I've written that will change your brain state. A pattern in my brain is encoded, stored, and transmitted in a language that we've both learned in our own separate ways, then decoded to a pattern in your brain. But your brain is quite different from mine, and you're free to implement any sort of sandboxing you want against this information, and even against the language specification itself, in your own head. Erasing this barrier is equivalent to erasing individuality, so I think we ought to be able to do this for digital programs as well.

I hope that in the not too distant future, we will be able to design and manufacture custom hardware on a per-person basis. Because that's what it will take to preserve human autonomy and bodily integrity in the upcoming era of AI and brain augmentations. I don't want my neural interface to have a hardware-level backdoor (https://en.wikipedia.org/wiki/Intel_Management_Engine) like my desktop computer. There is also a bigger evolutionary problem behind this, and I expand more on that in this tangentially related past comment: https://news.ycombinator.com/item?id=49690354

Regarding the biggest: nixpkgs sits at around 140k packages, way more than others.

Best: I still argue that Docker and Conda are more accessible, but my point was to go for reproducible, declarative science. Nix environments are exactly that. They're not perfect and are, as you point out, not standardized. But they cover much more ground that Docker. If you trust the upstream nix repo, you can get bit-level equality at every single build you (or anyone else) does. Docker relies on huge binary blobs that you can't inspect and that can be pretty much arbitrarily swapped around.

I'm totally fine if Guix takes over. Or the next big thing. As long as it's declarative and reproducible.

I'd love to hear you expand on how I cherry pick terms and definitions.

The Turing Way seems great and I'm all for education. One of the most interesting conversation topics in this thread is, to me, how to create the necessary incentive structures. Know how is only part of the way. We need to secure the credit assignment for "openness" both in academia and industry.

This seems to be strongly US-centric. In other (welfare) countries, publicly funded registries are anonymized and made available to research. For every single case. Of course, there are tons of data we don't see, but that shouldn't be an argument for not trying. The 99.99% unpublished cases is because our models/explanations/knowledge can't efficiently condense the medical mystery into a diagnosis code.

Everything can be prevented given sufficient knowledge. That's not the point. The point is how to prevent as much as possible.

Well said! And thank you for recognizing the effort.

The important part here is, as you say, to light candles and insist on rigor. Coincidentally, history tells us that that also gets us further. So by pure memetic selection, this strategy should win

I agree that a lot of the practices in academia are misaligned with the original goal. But can't you say that for other systems/institutions as well? Point being, what about keeping the scientific method as the north star - as a good heuristic to avoid BS arguments and awarding low-quality research. And, crucially, to stay sane. My post is pretty naive, but I stand by the ideal of pushing knowledge as reproducible models.

Of course, every area has similar issues. The main unique problem with (contemporary) academia is the one I mentioned:

> tech and industry, where producing junk actually has real negative economic and personal consequences

In academia, you can just endlessly produce low-quality garbage, and basically make a career out of this. In industry, things more often eventually at least have to work and survive contact with reality. Academia mostly lacks this basic check.

The scientific method should be the north star, sure. Much of what is happening in academia is cargo-cult / degenerate / pathological science though.

100%. I'm not saying the vision we're talking about is there yet, I just think it's an enticing thought. The existence of (predatory) publishers is a sad and miserable joke in its own right.

:-) I can comfort you with the fact that the text is not AI generated. I did use AI to proof read it and I liked some of its suggestions to improve the flow of the text. English isn't my first language, so this is a great help for me.