This assumes all layers of cybersecurity are broken - We call self-replicating software a virus, and we have protections against it. Same with stolen API tokens, just rotate them. Suspicious behaviour, nothing new, we have detectors for it. Stolen CPU / GPU cycles, we had that when crypto was a thing and before that when folding@home was cool, people were desperate to find more compute to the point of taking over systems. And we dealt with it.
A lot of the supposed risks / dangers are based on a supposition that cybersecurity is nonexistent or fatally, unfixably flawed and that AI agents are invisible. Neither of those is true.
I see your point but then if cybersecurity is the answer then what's the risk at all? An entire model copying itself somewhere would be found just the same as my hypothetical misbehaving agent.
I can imagine some people feel like it's cheating because they did it the hard way. I have much respect for those people and their skillset and I hope AI doesn't discourage them.
Yeah, I'm really pissed at all these car drivers doing marathons in 26 minutes because I've been working on it for years and can't get my time below 4 hrs.
Even though I used AI, it took me months to decipher parts of the emulation. I had to study, and I loved learning things I didn't know. It's also a very beautiful learning path; everything related to technology is magical. Engineers from all over the world are trying to revive the iPod and products that are no longer commercially available.
Yeah, same cables but different specifications is a danger; I've got old HDMI / DP cables for example, they're backwards compatible and still fit into modern systems of course, but won't do 4K / 144 hz (or whatever).
One day I'll sort the box out. There's even SCART and PATA cables in there, as well as my old diskette holder.
...I should try and build myself a vintage PC sooner rather than later. Mind you, my dad got rid of all our old shareware diskettes years ago, unfortunately.
Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.
LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
Age verification is not (should not be) the same as ID verification / storage / etc; the US needs laws similar to EU ones where companies can only get the minimum required PII. In the case of age verification that's zero, or a boolean value "yes this person is over 18" that they get from a trusted party like a bank.
This has not been the case at any of my past employers in B2C...
There was a fair amount of scrambling to get GDPR/CCPA compliant, but even that was done largely with a prevailing "ah, this is a defensible thing to store, make sure you can annonymize it or scrub it if needed" vs "stop storing this."
Starting with the ones that are most popular in the US, "Big Tech" usually includes:
- Google - Gmail and Maps contain massive amount of PII, Photos contains all sorts of other sensitive stuff, and they have not treated those aspects of those products like radioactive waste
- Meta - Facebook has a real names required policy even. Not a lot more needs to be said there, I think.
- Amazon - Nothing I've seen about trying to move away from how they need your name/address/payment info and all. If anything, more and more geographic targeting and such.
- Microsoft - Now you need to tie your local Windows install to their cloud services, not moving away from collecting user info. Also moving towards subscriptions which means PII and payment info.
- Apple - cloud accounts + email + payments + subscriptions all here too. Getting into banking-type services, that's leaning into PII...
- Netflix - more and more PII (IP tracking and geolocation combined with things like email and name) to fight account sharing...
> suddenly there are millions of euros of fines headed your way
I'm not sure about that. As I see it, there is no business case for treating PII carefully: security costs money while leaking PII costs nothing and has no repercussions.
The government isn't typically facilitating the check. In order to verify your age, you will be required to hand over your ID to a third party, who's privacy and security practices are likely: "Trust me bro."
Furthermore, I think many folks have reservations about requiring an ID checkpoint to utilize a computer. Obviously it's not that bad yet, but I don't think it's hyperbolic to state that the landscape is certainly trending in that direction, and it's absolutely not unreasonable to point out that governments and institutions to have a material interest in setting up access controls on who can and can't use the internet (read: participate in society).
Right, I'm not a fan of the ID check or identity/age verification. I'm just trying to understand the conspiracy theory kind of idea that "they're trying to get all our IDs!!!1!!" sentiment.
"The government already has our IDs" is missing the correlation between the ID and the service. They know your face and your name but not what you do on the internet. The website knows what you do on the internet but it isn't tied to your name. It's the tying them together which is the problem.
Which is why the people trying to do it always pull out the misdirect about ZK proofs. Those don't fix anything because a system that actually preserved privacy wouldn't be able to prove that the user is over 18, only that someone is over 18, not necessarily them. And that in turn means you're setting up a rug pull. You roll out a system which is indistinguishable from the perspective of ordinary people from the one that screws them, and then that system can't actually exclude minors so what follows is calls to change it to stop protecting privacy, at which point the people trying to collect everyone's ID will be arguing that you already have to show ID.
It also presumes you would even get a privacy-preserving implementation to begin with, which a pretty credulous assumption given how these things usually go.
It's not possible to prove the user is anything ever. It probably never will be. Likely what will happen instead is that the person in the ID will become personally liable for accounts verified with that ID. Which is even more reason to reject ID verification.
Governments don't necessarily have the connection between your ID and what you do online, though, and some governments are known to massively buy publicly available data from data brokers to circumvent existing laws. By "some governments" I mean the US government, by the way. That's not a conspiracy either, it's well-documented.
We don't want your static ID, we want where and what you are at this moment in order to better tune the algorithm. Your ID is frozen in time, so at any given moment, big tech knows more about you than the government does. IDs are only good for minimal verification purposes.
Well the EU's own government websites are all polluted with the cookie banners too so it's obvious that they can't even resist collecting visitor tracking data themselves.
Then, Chase knows you've verified your ID somewhere, the relay knows that some Chase user verified themselves at Pornhub, and Pornhub knows that the user is over 18, without knowing their identity or what bank they're using.
You could also do this with ZKPs and device integrity protection. The latter is more secure but more complex, the former is much simpler and openness friendly.
> You could also do this with ZKPs and device integrity protection.
How has the anti-competitive lock-in scam of "device integrity protection" entered the discussion? Using ZK proofs without it has exactly the same effect.
There are far too many attestation-passing insecure devices to expect attestation to have any security value against attackers who can choose any of those devices on purpose.
Or we could use multiple relays so nobody knows both the bank and the purpose, or even who would know the other piece of data... and now thats just Tor but for identity verification. Might be a good idea actually, except the whole point is Anthropic wants to know who you are.
I think the key here is “a trusted third party” more than “bank”.
Also, I’d rather a company know “he has an account at Bank of America” than “His full government name is Bit Masher and his driving license number is 9”
Well, they already know your credit card issuer, it's not that wild. Really Visa/MasterCard should offer age verification on their network... They have all the necessary components.
The EU is also doing age verification by showing ID, only difference is that you have to trust that their zero proof concept works and they're doing what they say they do.
In the US we just assume no one does what they say they do.
Not "a" bank. You (somewhat) trust your bank, I would imagine. Since you know, they have your money.
GP's hypothetical here is that Anthropic or other service provider who wants to do "age verification" could partner with (among others) your bank [1], where bank can answer yes/no to "is this user >= 18?", without revealing any other personal info to the SP. Allegedly.
[1] via an intermediary, no doubt. Trying to do a "full-mesh" of partnering of every SP with every bank directly would not scale.
You may trust them with your money, but does not equate to trusting them with anything else. Principle of least privilege, if you will. Anthropic has your chat data, which in many ways is more valuable than money, so if the only bar for free lying giving out your personal details is trusting a business with something of yours then why bother with this complex scheme and give Anthropic all of your personal information directly?
> Banks by law require substantial PII in order to even do business with you.
And those laws are extremely invasive and should be repealed. It's offensive to have a law that de facto requires you to identify yourself in order to pay for a newspaper subscription or buy contraceptives over the internet.
But that's not the issue in this case. It's that the bank knows your name and what you buy -- already very bad -- but now you want to create a path to tying that information to everything you do on the internet.
Gift cards don't allow reloading which consequently makes them a significant inconvenience to use for subscriptions. It's sort of like saying you can pay for something by going to their offices and paying in cash. Okay, but then why does the way that 99.9% of people are actually going to do it have to be the one that invades their privacy and puts everything they do in a database?
> Banks by law require substantial PII in order to even do business with you.
That is technically true, but keep in mind that in the early days of banking banks expected you to provide that information without the hand of the law requiring it. The laws you speak of came into effect after the fact to normalize across the industry what the banks were already doing. History is repeating itself in tech, and we already know lawmakers are waiting with bated breath to do their thing all over again just as they did in banking once critical adoption is there.
I expect what you are trying to get at is that you are willing to trust a bank because giving them your life story was already normalized before you were born, so you have never thought to question it. Whereas tech doing the same now feels new and scary. However, that's a funny way to look at it as the kids born in the future, who never knew the world where you could use the internet anonymously, will see giving tech their ID as being no different than how you see giving your bank your ID.
Although I can understand why you would now question why any business that does little more than store numbers on a computer needs a comprehensive profile on you by law or otherwise. Normalized does not equal sensible. That is a fair point.
> You may trust them with your money, but does not equate to trusting them with any other personal information.
How do you expect that they will give you your money when you walk in to a branch and ask for a withdrawal? Or that they'll replace a lost card? Surely you'd expect them to verify your identity.
They could verify your identity if you've given it to them.
But it would also be completely reasonable to authenticate the customer exclusively using mechanisms other than government ID. You can already make a withdrawal using your bank card and PIN. If you lose your card you could sign into their website using your password and request a new one etc.
Consider what happens if you lose your government ID. Your bank has much better ways to authenticate you at that point than the government does. Government ID has a major bootstrap problem, whereas patronizing a service doesn't because a new account with no money in it belongs to whoever is signing up for it regardless of who they are, and you can at that point give them a bank card and have them provide a password and email address etc. that allows you to identify them in subsequent transactions without ever needing their name.
Well, how do you expect a website to know that you are 18+?
The technical solution offered earlier was to have a trusted third-party only be willing to answer the "is this person 18+?" question. Of course, the same works for banks. The trusted third-party can answer "does this person own this account?" without needing to reveal to the bank any other information about you.
Now, that still leaves open the question of who you can trust. If you can trust a business run by people then that allows you to trust a bank, sure, but it also allows you to trust a tech company, so why not just give the tech company your ID and skip all that technical complexity? Understandably the broader idea presented earlier was that you cannot trust businesses operated by people, but then that includes banks, so...
For the sake of discussion, if we accept that technical solution and the need for a trusted third-party that is a business run by people, surely it should at least be a business that does nothing but offer profile trust to minimize the blast radius? For what reason would we want that business to have their hand in other activities like chat or banking?
> Now, that still leaves open the question of who you can trust. If you can trust a business run by people then that allows you to trust a bank, sure, but it also allows you to trust a tech company
Banks are subject to much stricter regulation than any tech company is or ever will be (IMO, I could end up surprised).
I trust my bank to know who I am and keep track of my money. I trust big tech to lose or misinterpret my data, to close my account for no reason, etc, and to face zero consequences for those failures -- not even fines.
> Banks are subject to much stricter regulation than any tech company is
That's true, but remember what we're talking about: Using banks (or another trusted third-party) to assert your age instead of having big tech collecting your personal information themselves. There is context here. Writing comments in a vacuum makes no sense.
In that context it is understood that only incentive for big tech to follow the proposed is regulation, but if you are going to push regulation on them then you can regulate them just as much as banks.
Your broader point that, in the real world, there isn't much political will to push any of that regulation is also no doubt true, but, again, writing comments in a vacuum makes no sense. The hypothetical of big tech adopting a third-party attestation system was already understood to be just that: hypothetical.
If I had a bank, I'd trust it too, it being mine. Most people don't have the luxury, though. In practice, they have to outsource banking to other people. And those other people are already quite likely to move around between both Anthropic and various banks in search of whomever will offer them the most economic benefits. It is not like a particular logo on their current business card is going to change their character. You do you, of course, you are already unique in having a bank you can call your own. But the fact remains that most people strongly believe that a person's trustworthiness is of the person, not the activity they happen to be doing at the time.
Strange, when I buy beer at Kroger and the "ID verification" guy has to come over, he looks at me and just keys in 01/01/1970 as my birth date most of the time. Or just asks me what my birthday is and I give him a random date in the late 1960s.
That specific link is basically just "having an account" if anonymity is turned off.
If you include the anonymous version are you calling a basic cookie an "advertiser's dream"? Then that's not something new they would get if the cookie tracked age verification. And you can reset cookies every visit (or more often). The "that person" of a site-specific session cookie is not a big deal for privacy.
Armchair / gut feeling opinion: using AI makes a developer feel like they can manage more code than they normally would. But it's a risk, if an agent can no longer make sense of the code, or the developer can no longer make sense of what the AI is saying, you're in trouble.
Not a new problem, of course - developers wrangling large amounts of code (more than they should or can) is a recurring challenge in the industry.
I'm inclined to agree on certain points; true native apps are, generally speaking, better when it comes to e.g. customer experience. It comes with risks and costs (but I think the cost vs benefit is exaggerated) - risk being the ability to find good native developers, which aren't as common as e.g. web developers who can switch to RN fairly easily.
But there's a factor few people (that decide on using RN) miss; it adds a layer of indirection, so you're no longer as "in touch" with the underlying platform. While possible, few RN developers would consider adding widgets or smart watch apps to their main app, but (I feel like) if you're a native iOS developer who is all-in on the Apple ecosystem, you're more likely to try and adopt these features (where applicable).
(disclaimer: RN is my current day job, used to do native iOS development and I frequently miss it)
When it comes to number of engineers, always keep in mind that it costs more engineers to maintain (and understand) software than the initial development push.
I’d add on to that lifecycle demands: a development project with no users has a tight cheap feedback loop with high quality bug reports with relatively obvious bug sources, once issues start working through layers of QA and customers the per-issue costs raise by several factors (50x to 200x are numbers I’ve seen from research, YMMV).
I would feel very confident slapping something onto Chrome ‘back then’ with only internal users. Today, with millions of users and CEO-level attention to screwups, I’d use a lot more time per issue. It’s a fundamentally different risk/reward/confidence situation.
> At some point it may be necessary to add the requirement for large companies to have a both human contact point and perhaps a requirement for specifics on account terminations and the like, or at least a way to request them, to the letter of the law.
A lot of the supposed risks / dangers are based on a supposition that cybersecurity is nonexistent or fatally, unfixably flawed and that AI agents are invisible. Neither of those is true.
reply