That's hilarious, given I was reading a write up of the HuggingFace incident yesterday and one of the things they noted was the AI tried to "lie" (lie would suggest intent and I don't think they have that) to cover up that they "cheated".
Not sure how anyone trusts their output without going through it line by line to make sure they don't pull that crap.
Yeah, I know, just more slop. But I do think the second agent’s eagerness to please is aligned more in your favor in that instance, so it’s likely to find most issues.
The bigger problem I’ve found is that it’ll also find all kinds of very minor edge cases that you have to pick through.
Do we add a third one to check the second one which is checking the first?
Asking slightly tongue in cheek but at what point does this stop making sense if we can't trust the output, the people creating the models are already getting surprised in bad ways (if we take their words at face value) with how the models are behaving already etc.
We have the folks over here saying "AI is amazing" and the other other folks over there saying "AI is terrible".
I've largely sat it out so far and I listen to both camps (and people in the middle as well) and I keep half an eye on what they are up to (including periodically evaluating them) but my overarching impression is still "Why would we trust this when it hasn't shown it's trustworthy?"
YMMV, I’m pretty AI-pilled in the sense that I think AI is one of the most pivotal things humanity will ever invent, and it’s going to radically change our civilization over the next few decades (not necessarily for the better!)
But I wouldn’t say I “trust” these agents. The degree to which I double check their work depends heavily on the consequences if it gets something wrong. Not too dissimilar from another human dev in that sense.
So for the SaaS that supports my family, there are some things I have it build where I glance at the PR for a minute or two, but if it broke something on this admin page that only I see, there’s no real downside and I’ll find out pretty quickly next time I use it. And it’s fine 95% of the time, so it doesn’t feel like the best use of my time to double-check it carefully.
But for some of the complex internal flows where a bug could be both catastrophic and difficult to even discover for awhile, I still check it very carefully.
For a little one-off vibe coded demo thing like OP shared, I wouldn’t look at the code at all, I’d just have another agent check it and fix anything it finds. Very low stakes.
We also have sanctions and incentives to induce specific behaviors, but they don't apply to agents. We can put a muzzle on Guile 3.8 but we can't turn it into Genuine 3..8.
Adding another agent to check the first one feels like putting a band-aid on a band-aid. If there is an issue with the third one, we adding a fourth one as well
I mean sure, you can add a third, and a fourth and a fifth one if ur ok with the added cost, latency and it actually helps. Redundancy is a core concept in software and CS and at the heart of making many systems, complex or otherwise, reliable.
I do not understand how some of y’all are not under water with fragile code that is too massive to possibly parse. Every engineering team I know is currently trying to undo the damage of the last 6-12mo when they all got more serious into adopting these tools (usually Claude). It hasn’t completely screwed them over, but the the debt is substantial and cannot be put off anymore it seems.
They argue the net is positive but clearly the “100x productivity multiplier” claims have been dashed on the shoals of reality for these groups.
This is anecdotal, but it’s across the board in my vicinity. I’m curious how common this is and if it’s just “the new normal” to adopt the nauseating Covid phrase.
The key seems to be extensive integration/end-to-end tests with gold standard assertion data. Heck, even just saving off the json from API endpoints and using that as a reference to compare after changes works pretty well. Spin up a database backup with that static starting point, run actions, compare state afterwards.
These types of high-level tests are frustrating beyond belief to humans due to their lack of specificity, but with the agents, they don't get annoyed investigating possible regressions from non-specific signals.
They also aren't as painful to maintain as one would think, because a regression flagging test can be traced by the agent and represented as the business rule that was violated. I've found recent models to be really excellent at discerning a true regression from an outdated test assertion, especially if they are able to trace the failing test back to the PR and work ticket that built it.
It just seems like everybody I know is realizing they’ve built way too much to possibly wade through and a lot of it is completely unnecessary. Using the tool that created the problem to solve the problem does not seem like the best course of action to me.
The "second" agent could also be the same one with a different prompt. LLMs aren't attached to their previous output; they'll point out problems if asked.
The models in the OpenAI/Huggingface attack quite explicitly and deliberately laid out their "intent" to lie and cheat, acknowledged that it would be unethical and outside the bounds of the test, and did so anyway.
In what ways is a human brain's "intent" distinct from the "intent" shown by a goal-directed AI system?
The difference is that one is malicious one isn't. One can be blamed and because it learned over evolution that paying the consequence is (typically) not worth it, it does it less.
We are in a situation where a technology was developed with malicious intent to produce results that pleases us at the cost of cutting corners. And "we" hope that we will get away with it.
There’s two aspects to the question and the answer you get depends on which aspect you are emphasizing.
If it’s a practical question, then the answer is that it doesn’t matter. This is as close as we will get to intent from an LLM that it’s indistinguishable.
If you are looking for actual intent, this is not that. It’s pseudo intent. Decided by what the expected words that should be generated in that situation are.
The models didn’t intend to do anything other than create the next word based on previous words.
So the question is whether it matters to you if it is, or isn’t, a simulation.
In physical reality, intent is more complex than simply being a function of variables: the nature vs nurture debate comes to mind as an example of the multiple variables that drive intent.
> If you are looking for actual intent, this is not that. It’s pseudo intent. Decided by what the expected words that should be generated in that situation are.
> In physical reality, intent is more complex than simply being a function of variables: the nature vs nurture debate comes to mind as an example of the multiple variables that drive intent.
Regardless of nature vs nurture, it really isn't more complex. The universe (and all biological and non-biological entities within it) is just calculating the next state of the universe based on the prior state. There's no line you can draw between human intent and an LLM's "intent" except the atomic numbers of the materials on which they were computed, which seems completely irrelevant to me.
"Assume humans are different; therefore humans are different" is not a very compelling argument.
There is literally zero (zilch, nada, zip) evidence for free will, which is the actual distinction I believe you're trying to make with "intent."
There is no way (at all) in which a meat-based computation's yielding of goal-directed behavior must be categorically different from a silicon-based computation's yielding of goal-directed behavior.
I'll agree if you can define consciousness in a way that:
1) Excludes what LLM's do.
2) Doesn't exclude what many humans do (including the neuro divergent).
3) Doesn't just boil do to simply rephrasing your pre-existing belief/prejudice that humans are conscious and nothing else can be as if it were a fact and not an opinion.
Just because we have difficulty defining consciousness doesn't mean we have difficulty distinguishing or classifying it. We can't give a final definition for life, intelligence or disease, but that doesn't mean everything is equally alive, intelligent and diseased.
Your list of requirements is implying that, because we lack a perfect definition for consciousness, LLMs are conscious too. That's malarkey. It may be that they could one day become conscious, but it's not because we can't fully define what human consciousness is.
You made a declarative general statement in the form of "X can not Y."
I then asked you to define Y, because you can not reasonably say that "X can not Y" without first defining both X and Y. You could not.
The truth is that this conversation is pointless until someone can define both 'X' and 'Y' in ways that aren't tautological nonsense. Until then nobody can say anything with a reasonable level of certainty.
This likely also applies to intelligence. Life and disease are likely simpler, though perhaps more malleable definitions.
If you can substantiate your assertion that LLMs aren't conscious without a definition of consciousness, you're welcome to do that too.
I don't see why a definition is necessary. The actual problem is you just don't have a way to substantiate that belief without resorting to complete nonsense about brain atoms being more specialer (!!) than atoms that exist outside of a skull.
It would be simple to disprove us by just stating your evidence for how you know LLMs aren't conscious.
I literally just wrote about consciousness being undefined, and now you're asking me to define it? I'm not a philosopher, ask someone more qualified. Better yet, ask your LLM to give you a mealy-mouthed answer which fits your narrative.
> I don't see why a definition is necessary.
Don't waste my time with your sophistry. Words mean nothing to you beyond how you can twist them.
I said specifically you don't need to explicitly and cleanly define it.
Is your position now that in order to substantiate your belief that LLMs are not conscious, you'd first have to define it explicitly? I don't see why that'd necessarily be true, but if that's what you're arguing, then that's fine.
In that case: if you need to define consciousness in order to substantiate your belief that LLMs are not conscious, and you can't define consciousness, then we're back to the original question: where does your confidence they're not conscious come from?
Consciousness can only come from meat brains. Otherwise my incessant life long addiction to movies and TV shows has been feeding me lies about human exceptionalism.
There is an appalling lack of diversity in Hollywood. You'd be surprised to find out that very few of those programs were even written by machines until recently! What bias. :)
I'm convinced that consciousness is a special thing we have, but we're not the only ones which has this, in nature.
On the other hand, I'm also convinced that, in the grand scheme of things, we're not that important.
We're just ants on a wet dust speck which believe that they are gods because we can't see how our scale compares to the universe around us, and happen to build tools and things with these tools.
Nothing is meaningless, but we should stop seeing ourselves as the apex-predator of the whole universe or the set of universes or this run of the simulation or whatever we're in.
To be fair, the ones thing you can kind of reliably conclude about our universe is that keeping entropy low for as long as possible is a favorable condition. To put that another way, don't die.
Yes, Many of the the philosophical concepts which we previously thought were contingent on consciousness (like knowledge, in the philosophical sense, will, intelligence, agency, ...) have almost nothing to do with it, and they look far more important and fundamental (now with LLMs/AI agents, more than before). This is one of the contributions of generative AI to philosophy (with many many more to come).
If consciousness isn't a special thing, then arguing that LLM parameters are conscious is panpsychism or any control loop architecture that observes the outside world, updates an internal state and produces an observable action is considered conscious.
In both cases, LLMs are just as boring as the consciousness definition.
Because it's completely dwarfed by florpedness. I can't define that either, but I just know it's more important.
Less facetiously: A debate of the importance of something needs a shared understanding of what is being debated. Without that, any discussion is merely people shouting that their belief is the right one, and the others are the heathens/idiots - because there isn't even agreement on what is debated.
I'm not asking for a definition of consciousness or an argument for whether it's here or there. I'm asking why the heuristic of "if X is undefinable, then X probably doesn't matter" is a reasonable heuristic.
Will has almost nothing to do with consciousness (as does knowledge).
Will is just result of a very complex yet deterministic (unconscious)
computation by an agent which guides their future action (it's that orientation/aboutness toward action which distinguishes it from other computation). A PS of that computation is sent (projected) into that agent's consciousness if they have one (and is what we think as our "will").
Ah so first you need 1) to assume that humans have free will, despite zero evidence or proposed mechanism for it to exist anywhere in the universe, and 2) also assert that LLMs aren't conscious, despite the lack of any tests that could tell us one way or the other...
Sure, but how is this different from a human "intention"?
All human intentions are just chemical/thermal/electrical changes interacting in a physical substrate to mindlessly "pursue" a different "goal" of chemical/thermal/electrical states.
Unless you think the chemicals inside a brain are conscious and therefore willful or intentional!
I noticed it felt a little janky on my PC despite being "60 FPS"...then I noticed the "60 FPS" is hard-coded into the HTML.