Safety Isn’t a Stop Sign
Building AI that protects children without making them afraid to ask.
A Real Design Problem
Imagine a thirteen-year-old asks an AI:
“I think I might like someone. Is that normal?”
A very cautious system might respond:
“This is something you should discuss with a trusted adult.”
That response isn’t necessarily wrong.
But imagine receiving it every time you ask something sensitive.
Eventually, you learn something about the system:
“I can’t talk to this AI about things that actually matter to me.”
So you stop talking.
Now consider the opposite.
An AI responds freely to every question about relationships, sexuality, emotions, or growing up without considering the child’s age, intent, developmental stage, or level of risk.
That can also be dangerous.
This creates a difficult problem:
If an AI is too restrictive, children may leave. If it is too permissive, children may be exposed to inappropriate or harmful guidance.
We don’t think either extreme is good enough.
And this is where building Sisbot became much more complicated than simply adding safety filters.
The Big Idea
We used to think about safety mostly as a boundary:
Safe → answer.
Unsafe → refuse.
Real conversations aren’t that simple.
A child’s question doesn’t exist in isolation.
The same topic can mean very different things depending on:
the child’s age,
developmental stage,
intent,
emotional state,
context,
potential risk,
conversation history,
and how confident the AI is about its understanding.
So instead of asking only:
“Is this topic allowed?”
we need to ask:
“What is the safest and most helpful way to respond to this particular child, in this particular situation?”
Sometimes the right response is to answer.
Sometimes it is to ask another question.
Sometimes it is to provide a carefully constrained answer.
Sometimes it is appropriate to encourage the child to involve a trusted adult.
And sometimes the situation requires a much stronger safety response.
The important difference is this:
Safety doesn’t always mean ending the conversation.
Sometimes safety means knowing how to stay in it.
What Research and Child Development Tell Us
Children are not all at the same stage of development.
Age matters, but age alone doesn’t tell us everything.
Children develop at different rates. Their understanding, emotional maturity, ability to assess consequences, and capacity to make decisions can vary considerably.
That means a Child-Aware AI cannot treat “child” as one homogeneous category.
This is one reason we think age should be an important signal—but not the only one.
The system also needs to consider what the child is trying to accomplish and what risks might exist in the situation.
This is particularly important for sensitive subjects.
A question about relationships might simply be curiosity.
It might be about friendship.
It might be about emotional confusion.
It might involve pressure from another person.
The topic alone doesn’t tell us which one it is.
Context matters.
The Risk of Being Too Safe
This is one of the uncomfortable things we’ve had to think about while building Sisbot.
What happens when safety interventions become so restrictive that children stop using the system?
Imagine that every difficult question receives the same response:
“Please talk to a trusted adult.”
Sometimes that is exactly the right advice.
But if it becomes the default response to every sensitive subject, we may unintentionally teach children that the AI is unavailable whenever a conversation becomes difficult.
And children don’t stop having difficult questions just because an AI refuses to discuss them.
They may simply search somewhere else.
That could mean less reliable information, less context, and potentially less safety.
This doesn’t mean an AI should avoid boundaries.
It means that boundaries need to be purposeful.
The goal isn’t to keep the conversation going at any cost.
The goal is to avoid leaving the child alone with a difficult question when a safe, age-appropriate conversation is still possible.
From Rules to Calibration
This led us toward a different way of thinking about guardrails or safeguards.
Instead of a simple list of topics that are allowed or blocked, we’re exploring an adaptive approach.
Think of it as a continuous calibration.
The system considers multiple signals:
Age
Who might this child be developmentally?
Risk
What could go wrong if the AI responds incorrectly?
Intent
What is the child actually trying to understand or accomplish?
Context
What has happened in the conversation?
Emotional state
Does the child appear distressed, frightened, confused, or vulnerable?
Confidence
How certain is the system that it understands the situation correctly?
These signals don’t produce a single universal answer.
They influence the level and type of guidance that is appropriate.
We sometimes describe this idea informally as:
Age × Risk × Intent × Context × Confidence → Adaptive Guidance
It isn’t a mathematical formula that magically solves the problem.
It’s a design principle.
The point is to make the system reason about the situation rather than simply react to keywords.
What We’re Trying in Sisbot
This is where our work becomes an engineering problem.
We’re exploring different levels of intervention rather than treating every sensitive conversation identically.
Stay and answer
When the question is appropriate and the risk is low, the system should be able to have a normal, useful conversation.
Stay and clarify
When the situation is ambiguous, asking another question may be safer than making assumptions.
Stay but constrain
Some conversations can continue, but the response needs stronger age-appropriate boundaries.
Encourage additional support
When involving a trusted adult or another appropriate person would genuinely improve the child’s safety or wellbeing, the system should make that option visible.
Escalate
When there is serious or immediate risk, stronger safety measures may be necessary.
The important part is that “trusted adult” isn’t simply a button we press whenever a topic becomes sensitive.
It should be part of a contextual decision.
Otherwise, we risk creating a system that is technically cautious but practically unusable.
The Child Behind the Topic
This becomes particularly important with sensitive subjects.
Consider a child asking about relationships.
The system shouldn’t only see:
RELATIONSHIP
It should try to understand:
Who is asking?
How old might they be?
What are they actually asking?
Are they looking for information, reassurance, or help?
Is there pressure or coercion?
Is there a significant safety concern?
What does the previous conversation tell us?
How confident are we in our interpretation?
A good response isn’t determined by a keyword.
It’s determined by the situation surrounding the keyword.
This is one of the reasons we believe Child-Aware AI needs contextual reasoning rather than increasingly long lists of forbidden words.
Safety and Trust Are Connected
This also brings us back to something we discussed in our earlier article, “Trust Is Reciprocal.”
An AI shouldn’t try to become the most trusted person in a child’s life.
But it also shouldn’t become a system that children learn to avoid whenever something important comes up.
There is a delicate middle ground.
The AI should be trustworthy enough that a child can ask difficult questions.
But humble enough to recognize when it isn’t the right place to handle the situation alone.
Sometimes the most responsible response is:
“I can help you think about this.”
Sometimes it is:
“I’d like to understand a little more before I answer.”
And sometimes it is:
“This sounds serious, and I think getting a trusted adult involved could help keep you safe.”
The challenge is knowing which moment calls for which response.
What We Still Don’t Know
We’re still building this.
We don’t have a perfect answer for adaptive safety.
Some of the hardest questions remain open:
How accurately can an AI infer a child’s age or developmental context without being intrusive?
How should the system distinguish curiosity from harmful intent?
How much context is enough before the system becomes overly invasive?
When does encouraging a trusted adult become counterproductive?
How do we measure whether a safety intervention actually improves safety?
How can we detect when overly restrictive responses are causing children to disengage?
How should confidence in the system’s own interpretation affect its response?
Where should the boundary between continued conversation and escalation sit?
These are not just policy questions.
They are engineering questions, product questions, ethical questions, and research questions.
And we don’t think they should be answered behind closed doors.
Join the Discussion
One of the reasons we’re documenting Sisbot publicly is that we don’t believe any single team has all the answers.
We’re experimenting.
We’re changing our assumptions.
We’re building guardrails and then asking whether those guardrails actually produce the behavior we intended.
Sometimes the safest-looking solution may create a new problem.
Sometimes allowing a conversation to continue may be the safer choice.
That tension isn’t something we want to hide.
It’s one of the problems we want to understand better.
So here’s the question we’d love to hear from you:
How should an AI protect children without making children afraid to ask for help?
If you have experience in child development, education, AI safety, product design, psychology, engineering, or simply have a perspective on what children need from technology, we’d love to hear it.
Because perhaps the future of child safety in AI isn’t about building taller walls.
Perhaps it’s about building systems smart enough to know when a wall is needed—and when a child simply needs someone to stay and listen.