It used to be that people recognized that one of the unintended, unfortunate limitations of natural-language interfaces was that it was hard for users to discover their range of capabilities. Now we're designing that flaw into them on purpose.
Every single computer systems I've designed, built and shipped into production has limits programmed into them or else they will be abused or work incorrectly, why should an LLM be any different?
We still need to program the computers, it's just that now we're trying to (somewhat unsuccessfully , see jailbreaks) using the English language to program computers.
I'd love for you to moderate anything even close to controversial. You'd quickly figure out specific rules go to shit very quickly on internet based services covering wide geographical areas. You'll figure out there are a lot of definitions of what the word 'is' is, and an unlimited number of actors willing to waste all of your time arguing about it.
Even HN has 'gray' rules because dealing with assholes is difficult.
Ok I understand what you're saying, you'd like full transparency into how the limitations are configured. However I'd have to reiterate, I wouldn't normally publish this information because it increases the chances of workarounds being discovered in case my solution isn't bullet proof. I'd say the same goes for OpenAI.
> [...] I wouldn't normally publish this information because it increases the chances of workarounds being discovered in case my solution isn't bullet proof. I'd say the same goes for OpenAI.
This is commonly known as "security through obscurity"[1] and has been shown to be ineffective most of the time.
Thanks for the link. I'm very familiar with this already though.
I don't rely on obscurity for 'security', i just don't think implementation details are required for most users so I don't publish them.
I'm very familiar with security through obscurity, ultimately I like to think the systems I build are secure but I can't always be sure, so why give people a head start? Not publishing details gives me time improve security.
Security through obscurity might not be the best approach, but you should know it's fairly common. For example when I generate a link to a Google Doc and "only those with the link" can access the document, I think that's a form of seurity through obscurity. No one is going to guess the link in any practical time frame...
I totally get this, since we (collectively) are still trying to figure out how to "program" LLMs. There is definitely a risk that too much transparency leads to attacks.
At the same time, security by obscurity does not work in the long run. In fact, the existence of this repo of reverse engineered prompts maybe means that secrecy is impossible.
Even worse, we won't necessarily know when the information leaks out, so we don't even know what compromises are out in the wild.
…which is equal parts annoying, bad for usability, and snake oil for security. Put SSL on 443 with proper ciphers, and tell me what your bloody AI shouldn’t be used for, for chrissakes.
The first rule of a business is to not do anything illegal before serving their customers.
It gets even tricker in this case because you're exploring new territory. What OAI chooses to do here can and likely will influence laws in the near future.
If an angry wife finds a guy who she can pay $100,000 of her husband's money and he promises her she'll be set for life, and she does it, who's the evil one?
