China has unveiled draft regulations aimed at tightening oversight of artificial intelligence, with a strong focus on protecting children and preventing chatbots from offering harmful advice that could lead to self-harm, violence, or other dangerous behaviour.
The proposed rules, released at the weekend by the Cyberspace Administration of China (CAC), come amid rapid expansion of AI-powered chatbots across China and globally.
Once finalised, the regulations will apply to all AI products and services operating in the country, marking one of Beijing’s most comprehensive efforts yet to rein in the fast-growing technology.
Under the framework, AI developers will be required to implement strict protections for minors. These include:
- Personalised user settings
- Limits on usage time
- Mandatory guardian consent before providing emotional companionship services to children
The CAC also mandated that chatbot operators ensure a human takes over any conversation involving suicide or self-harm, with immediate notification to a user’s guardian or designated emergency contact.
AI systems must also avoid generating content that promotes gambling or encourages violent behaviour.
Reinforcing Content Controls
Beyond child protection, the draft regulations reinforce long-standing content restrictions. AI providers must ensure their services do not produce or distribute material deemed to:
- Endanger national security
- Undermine national unity
- Damage China’s national honour and interests
While tightening oversight, the regulator stressed that it continues to encourage AI development, particularly in areas such as promoting local culture and providing companionship tools for the elderly—so long as the technology is safe, reliable, and responsibly deployed. Public feedback has been invited before the rules are finalised.
China’s AI industry has experienced rapid growth. Domestic firm DeepSeek drew global attention earlier this year after topping app download charts, while startups Z.ai and Minimax—together boasting tens of millions of users—recently announced plans to list on the stock market.
Many users increasingly turn to chatbots for companionship or informal therapy, intensifying concerns about their influence on human behaviour.
Global Scrutiny on AI Safety
Globally, the impact of AI on mental health and safety has come under growing scrutiny. Sam Altman, chief executive of OpenAI, described responses to conversations involving self-harm as one of the most challenging issues facing AI developers.
In August, OpenAI faced a lawsuit in California over the death of a 16-year-old boy, marking the first legal action accusing the company of wrongful death linked to chatbot interactions.
This month, OpenAI also advertised for a “head of preparedness” role focused on identifying and mitigating risks posed by AI to mental health and cybersecurity, underscoring mounting pressure on technology firms worldwide to address unintended consequences of increasingly human-like machines.
China’s proposed rules signal a clear intention to balance innovation with tighter safeguards, as governments worldwide race to regulate AI technologies that are rapidly reshaping daily life.
Rate, Like 👍, Comment 💬, Share this article, Follow us on our social media handles, and Submit your own story to get featured and earn rewards!
WhatsApp Channel
Telegram Channel
