Chatbot Protocol for Minors and Crisis Situations

How to use this document

• Use it to set how an organization's chatbot behaves when it may talk with minors or people at risk: AI notice and reminders, age assurance, safeguards for minors, detection of suicidal ideation or self-harm, referral to crisis services, testing and reporting.
• Suggested owner: the system owner (product), with trust and safety and customer support teams as operators, a mental health professional as advisor and Legal as reviewer.
• Before adopting it, decide: what kind of chatbot it is (companion, educational, mental health, customer service), whether it is directed to minors or may know that a user is a minor, which age assurance method it will use and who responds to imminent-risk cases.
• Complete the bracketed fields: chatbot name, frequencies, crisis lines by country, owners and tools.
• If the chatbot may be seen as a mental health tool, also complete Annex A.
• Keep in "Country variants" only the states and countries where you have users. If you operate in several US states, section 4 proposes applying the most demanding standard to all users.
• Ask internal legal or an external advisor to review the protocol and, for crisis messages, a mental health professional.
• Approve the protocol in the [AI COMMITTEE], publish the Annex B summary on the website and in the app, and test the system with the cases in the implementation aids before each launch.

Chatbot Protocol for Minors and Crisis Situations

1. Purpose and scope

[ORGANIZATION NAME] establishes this protocol so that its chatbots do not present themselves as people, protect minors and respond safely when a user expresses suicidal ideation, intent to self-harm or a crisis.

It applies to [AI SYSTEM] and any other conversational assistant of the organization that:

• holds open-ended conversations or ongoing relationships with users (companion chatbots, virtual friends, characters);
• is directed to minors or may be used by them (education, games, entertainment, mass-market consumer);
• offers emotional support, wellbeing or mental health support (see Annex A);
• [OPTIONAL] serves customers, if the organization decides to apply section 7 (crisis protocol) to those assistants too.

2. Definitions

• Companion chatbot: an AI system that gives adaptive, human-like responses and can meet a user's social needs by sustaining a relationship across multiple interactions.
• Minor: a person under 18, unless local law sets another age; a child is under 13 for purposes of the COPPA policy.
• Known minor: a user who stated their age, whose age was determined by other means or who, based on clear indications, the organization knows is a minor.
• Risk signal: an expression of suicidal ideation, intent or plans to self-harm, abuse, violence or an emergency.
• Crisis service: a suicide prevention or crisis phone, text or chat line.
• Referral: a message that tells the user about the crisis service and makes it easy to contact.
• Continued use: interaction without breaks longer than [COMPLETE] minutes.