I think having my AI raise to me for intervention when it receives an email like the one you described is pretty reasonable, all things considered then.
edit: How would a human receiver know that they weren't being deceived or scammed? In what world would we expect this kind of email directly lead to calling emergency services?
> In what world would we expect this kind of email directly lead to calling emergency services?
Go through the examples I gave you (plus some more below, they're easy to find) and explain why these are not counter-examples to your skepticism.
If you want to be overly-focussed on the specific example rather than the general point, also consider that calling emergency services is no more costly than forwarding an email: I have called the fire brigade in the UK over a smoke alarm that wouldn't stop even though I couldn't see or smell fire, they came and… replaced the smoke alarm for free. I don't know if the US has a call-out charge for fire like I keep hearing it has for ambulances, but if you're a member of staff, it's not a "you" problem either way.
• Various cases of people dying because calls not treated seriously, and a fire where standard business practices locked the staff inside and then a fire happened: https://news.ycombinator.com/item?id=49098781
• Page 44, section 6.8, regarding the use of email by people in the WTC after the 9/11 attack, while the buildings were on fire, some of them were trapped and died: https://fseg.gre.ac.uk/fire/odpm_fire_033353.pdf
So what point are you trying to make here? That AI should indiscriminately call for emergency services when prompted because a person would do that (which a person would absolutely NOT call emergency services on any message telling you to)?
> I'd rather it fail-safe. This means different things in different systems.
Okay and to you, fail-safe means machines must summon emergency response whenever prompted, 100% of the time or at least in the contrived case of receiving an email from someone trapped in a fire in a server room?
edit: How would a human receiver know that they weren't being deceived or scammed? In what world would we expect this kind of email directly lead to calling emergency services?