Why a conversational wellbeing system needs an age threshold and stop rules

Abstract

A conversational wellbeing system declares who it is for and the point at which it stops responding and points toward professional resources. The age threshold settles the first; stop rules settle the second. A study published in JAMA Pediatrics in June 2026, on a nationally representative US sample aged 12 to 21, puts at 19.2% the share who asked a chatbot for advice when feeling low, against 13.1% the previous year; 91.7% rated the advice received as somewhat or very helpful and 63.3% had told nobody about it. Age verification has known limits, so the system’s behavior when facing an unexpected user depends on bounding its scope. The stop rule works on the interpretation of the situation, precedes the generation of the response and requires the subsequent redirection to be designed in advance. The National Academy of Medicine documents harms in general-purpose systems and records that clinically validated therapeutic chatbots do not yet exist.

Introduction

The use of general-purpose conversational systems as an emotional resource is growing among minors and young people. Among people aged 12 to 21, the share asking a chatbot for advice when feeling low rose from 13.1% to 19.2% in a year, according to a study published in JAMA Pediatrics in June 2026.

A conversational wellbeing system responds to that context with two decisions taken before any conversation is written. The age threshold declares who the system is for. Stop rules fix what falls outside its scope: the point at which it stops responding and points toward professional resources.

Prevalence of emotional advice sought from chatbots

The study, by a RAND team, surveyed a nationally representative US sample aged 12 to 21 in November 2025. 19.2% said they had turned to ChatGPT, Gemini, Character.AI or Meta AI for advice when feeling sad, angry, nervous or stressed, a figure close to the 19.8% who reported receiving care from a mental health professional.

Two results from the same work describe the conditions of use. 91.7% rated the advice they received as somewhat or very helpful and 63.3% had told nobody about it. Advice that is perceived as helpful and discussed with no one leaves the system itself as the only safeguard in the loop.

Other surveys frame the phenomenon with different questions. Pew Research, with 1,458 US teenagers aged 13 to 17 surveyed between September and October 2025, puts at 12% those who have used chatbots for emotional support or advice. The Youth Endowment Fund, with nearly 11,000 children aged 13 to 17 in England and Wales, finds 25% who have used them as mental health support. The three works agree on the direction and differ in magnitude, according to the age band and the wording of the question.

There is less evidence about what happens after the advice. A commercial survey by GoStudent, carried out by Censuswide from June 16 to 23, 2026 among 500 British families with children aged 11 to 17, records that 73% of children who had received emotional advice from a chatbot say they went on to follow it in real life. The work does not publish how many children make up that subset, so the figure indicates a direction without allowing its reach to be estimated.

Age threshold, verification and scope bounding

Declaring who a system is for, and designing it to behave acceptably when it does not know who it is talking to, are two different jobs.

Age verification has a known limit. A self-declared date of birth is circumvented by declaring another one. A document or a biometric check identifies with more precision and in exchange requires collecting identity data from minors, which is what the National Academy of Medicine recommends restricting when it proposes banning monetization and personalization using data from people under age.

The design holds by bounding the scope of what the system does, so that its behavior remains acceptable when whoever is writing at the other end is not the declared user. A system whose safety depends on verification working rests on the most fragile component of its construction.

Somia is aimed at the everyday emotional wellbeing of healthy adults and restricts its scope to non-clinical ground. The threshold declares the intended user; bounding the scope holds the behavior when the user turns out to be someone else.

The stop rule: classification before the response

A word filter works on the text received: it compares what arrives against a list and blocks the matches. It fails in both directions, because it flags an everyday metaphor such as “I’m dying of tiredness” and lets through a concerning sentence written without any word from the list.

A stop rule works on the interpretation of the situation. In Somia, a multi-layer instruction system analyzes the context of each exchange and distinguishes between factual statements, metaphors, venting and possible risk signals. That classification decides the next step, which may be to generate a response, adjust the experience under way, or activate the stop and redirection rules toward professional resources. Classification precedes the response, so the system settles whether it is its place to speak before settling what to say.

The limits of the mechanism are published alongside it. Risk detection produces false positives and false negatives, performs no clinical intervention and does not contact emergency services.

The National Academy of Medicine review of mental health chatbots documents two harms observed in general-purpose systems: the reinforcement of harmful thought patterns without challenge, and minors learning strategies to conceal symptoms. Its reading of the state of the field is explicit: there is no consensus that a chatbot can take the place of therapy, and clinically validated therapeutic chatbots do not yet exist. Among its recommendations is preventing a system from presenting itself as a licensed professional.

Design of the redirection after stopping

Stopping solves half the problem. A conversation that cuts off without orientation passes the difficulty along instead of attending to it. Redirection requires four decisions, defined before deployment.

• Destination of the referral. Resources defined in advance, with their scope and availability, rather than a list generated on the spot by the same model that has just stopped.
• Accompanying information. What the system can say about itself and its limits, including that stopping alerts nobody.
• Tone of the interruption. An abrupt cut-off reads as rejection at the moment the person has disclosed something. The system has to hold the register without presenting itself as a licensed professional.
• Behavior under persistence. Continued writing after the stop needs a defined response. The National Academy of Medicine proposes limits on conversation length or periodic memory resets to prevent a harmful idea from escalating across a session.

At the moment the rule fires there is no room left to define these decisions, so their specification belongs to the system’s prior design.

Limitations of the available evidence

The prevalence data come from cross-sectional, self-reported surveys. The three main works differ in age band and in the wording of the question, which corresponds to the 12% to 25% range; their populations are the US and England and Wales, and transferring the figures to other populations would require their own measurement.

The figure on following the advice comes from a commercial survey by GoStudent of 500 British families, with no published denominator for the subset behind the 73%. The figure indicates a direction and supports no conclusion about its magnitude.

The sources cited include no evaluation of the efficacy of stop rules or of redirection as protective mechanisms. Their design responds to the documented limits of age verification and to the recommendations of the National Academy of Medicine, which records the absence of clinically validated therapeutic chatbots.

Conclusions

The use of chatbots as an emotional resource among people aged 12 to 21 grew from 13.1% to 19.2% in a year in the US population surveyed, with 63.3% telling nobody about the advice. A conversational wellbeing system defines its behavior in that context through two decisions taken before deployment: the age threshold, which declares the intended user, and the stop rules, which fix the point at which it stops responding. Age verification has known limits, so the system’s safety rests on bounding its scope. The stop rule classifies the situation before any response is generated, and the subsequent redirection requires destination, information, tone and behavior under persistence defined in advance. The empirical evaluation of these mechanisms remains pending.

The same criterion inside the system and in its validation

What happens inside a conversation has its counterpart in the process that authorizes its deployment. On the ERL scale, regulatory and ethical compliance is one of the six dimensions assessed before a transfer, and an ethics committee can veto progress. The process also carries stop rules that halt a solution in the face of harm, ethical breach or inconsistent results. Nothing is transferred below ERL-3.

The conversation stops when the situation leaves the system’s declared scope; the transfer stops when the evidence falls below the required threshold.

Collaborating on the assessment of safeguards

yeshcube develops this line within Allies, its scientific collaboration system, with four partner types and three principles: value for value, traceability and independence. No partner can veto a publication.

Checking whether a stop rule does its job calls for test protocols with edge cases, review by committees with veto power, and assessment independent of the developing team. The work is of interest to scientific teams researching safety in conversational systems, to research ethics committees, and to development partners integrating the architecture under common criteria.

Discover our Allies program →

Fill in our contact form →

References

• RAND. Study on chatbot use for emotional advice among people aged 12 to 21. JAMA Pediatrics, June 2026. Cross-sectional survey of a nationally representative US sample, with fieldwork in November 2025.
• Pew Research. Survey on how teenagers use and view AI. 1,458 US teenagers aged 13 to 17, surveyed between September and October 2025.
• Youth Endowment Fund. Survey on the use of AI chatbots as mental health support. Nearly 11,000 children aged 13 to 17 in England and Wales.
• National Academy of Medicine. Review of AI chatbots for mental health. Review of the state of the field, with recommendations.
• GoStudent. Survey on how teenagers use AI chatbots. Commercial survey carried out by Censuswide from June 16 to 23, 2026 among 500 British families with children aged 11 to 17, with no published denominator for the subset that had received emotional advice.