Governing AI Before We Have to Learn the Hard Way

What the “horseless carriage” experience can teach us about the adoption of AI

Artificial intelligence (AI) is rapidly expanding and becoming a part of our everyday lives. This includes behavioral healthcare. It started with documentation assistance (aka “scribe”) and is expanding into chart synthesis, quality review, compliance monitoring, clinical decision support, treatment planning, and the identification of patterns across longitudinal records.

The adoption of these technologies brings up a central question: what do we need to put in place to ensure safety and preserve human clinical judgment?

There is a useful historical parallel: the horseless carriage.

When the automobile came on the scene, we needed to do more than distribute cars for people. It was a transformative technology, but the automobile alone did not create modern transportation. Over time, we had to build the infrastructure around it: stoplights, crosswalks, insurance, traffic laws, and safety standards. Pedestrians had to learn to look before crossing the street.

Many of these protections were developed only after experience demonstrated why they were necessary.

Behavioral healthcare has an opportunity to approach AI differently.

Competency Must Evolve Alongside Technology

New technology tools don’t change the underlying aim in behavioral health: effective clinical care. Competency requires an understanding of the clinical implications of using that technology appropriately.

Generating an AI note is not the same as generating a good note. Clinicians need to recognize that an output may look well-written yet contain inaccuracies, inappropriate conclusions, omitted context, or language that extends beyond their professional scope. We know that AI is not perfect. Humans need to recognize a good note and edit a poor one.

And, most importantly, the human clinician is still ultimately responsible for the content under their name.

With automobiles, we decided that people need to obtain a driver’s license before they could drive a car. As any AI tool is implemented, a key question is what kind of skills do you need to use that tool?

There are different driver’s licenses for different roles. A truck driver needs to know more than people who have cars. Same goes in behavioral health, where a therapist, medical provider, clinical supervisor, quality leader, or compliance professional might need different competencies.

Clinical Judgment Must Remain Explicit

We might get self-driving cars some day. We will never get self-driving behavioral health. AI assistance cannot replace human clinical judgment.

AI can organize information, identify relationships within a record, surface inconsistencies, draft documentation, and suggest areas that may warrant consideration. These capabilities can substantially reduce administrative burden and improve access to information.

But the clinician brings something fundamentally different to the process.

Clinical judgment incorporates professional training, therapeutic context, observation, experience, ethical responsibility, knowledge of the individual, and information that may never be fully represented in structured data or a transcript.

Human oversight should therefore be more than a requirement to click “approve.”

Well-designed workflows should create meaningful opportunities for clinicians to evaluate, modify, reject, and add their own reasoning to AI-generated content.

The objective is not simply to keep a human somewhere in the process. It is to preserve the clinician’s role as the accountable decision-maker.

Organizations Need Rules of the Road

Early automobile traffic was governed largely by individual judgment. There were no lanes, crosswalks, or stop signs. You just drove where it made sense. As the number and speed of vehicles increased, we needed rules to promote safety and efficiency.

Behavioral health organizations face a similar challenge with AI.

Without organizational standards, clinicians may develop their own approaches to when and how AI is used, how thoroughly outputs are reviewed, what information is appropriate to generate, and when AI recommendations should influence clinical decisions.

That variability creates risk.

Here’s a risk that is happening now: clinicians are using AI tools today, even if they aren’t sanctioned or monitored by the organization. Large language models (e.g., Claude, ChatGPT) are becoming standard for the public. Tech-forward clinicians are eager to adopt their time-saving features. This creates risk, especially with privacy and security.

Governance should establish clear expectations regarding approved use cases, required levels of human review, access to clinical information, documentation standards, escalation procedures, scope-of-practice considerations, and circumstances in which AI should not be used.

These guardrails should not be designed to eliminate professional discretion. Their purpose is to create a reliable environment in which professional discretion can be exercised appropriately.

Accountability Requires Traceability

When a car accident happens, it is important to know what happened and who is responsible. We want to ensure the right party takes responsibility and we want to improve the system.

One of the traditional challenges of AI is the “black box”, where the source of the information in an AI output is unclear.

It’s important to know why the scribe portrayed a part of the session in a particular way. What is the source of decision support information?  What organizational standards or rules were applied? Did the clinician edit or reject the output?

This has implications well beyond technology governance.

Behavioral health documentation supports treatment decisions, medical necessity, utilization review, reimbursement, quality measurement, regulatory compliance, and continuity of care. As AI becomes part of the process that creates or evaluates that documentation, traceability becomes part of clinical and organizational risk management.

AI Requires Continuous Quality Oversight

Governance cannot end when implementation is complete.

Healthcare organizations already understand this principle in other areas of clinical practice. We monitor outcomes, conduct chart reviews, evaluate adverse events, identify patterns, supervise clinicians, and modify processes when evidence suggests they are not performing as intended.

AI should be subject to the same quality discipline.

Organizations should evaluate performance across programs, disciplines, populations, documentation types, and use cases. They should examine where and how often clinicians  modify AI-generated content, whether certain errors or omissions recur, whether recommendations remain within appropriate scope, and whether the technology continues to reflect organizational standards as those standards change.

This is analogous to the role of vehicle inspections. Initial safety does not establish permanent safety.

Clinical AI requires ongoing evaluation because both the technology and the environment in which it operates will continue to change.

Governance Is a Clinical Function

Cars are interesting: sports cars, safe cars, muscle cars, pickups, or electric cars. Yet, transportation is not just about the vehicle. It’s also about building roads and implementing laws.

AI governance is sometimes framed primarily as an information technology or compliance responsibility.

In behavioral healthcare, that framing is too narrow.

The most difficult questions are often inherently clinical: When is a recommendation appropriate? What constitutes sufficient information to support it? How should professional scope influence an AI-generated response? When should the technology defer rather than answer? How should uncertainty be communicated? Where must clinical reasoning remain explicitly human?

These decisions require multidisciplinary governance, including clinical leadership, quality and compliance, operations, technology, privacy and security, and frontline practitioners.

Clinical leaders, in particular, have an important responsibility in defining the boundaries between assistance and judgment.

The more capable AI becomes, the more important that distinction will be.

The Tea

The lesson from the automobile is not that transformative technology should be feared. It is that transformative technology eventually requires an infrastructure proportionate to its capabilities.

Driver competency, safety standards, traffic laws, inspections, accountability mechanisms, and oversight did not diminish the value of the automobile. They helped make widespread adoption possible.

Behavioral health is now beginning to build the equivalent infrastructure around artificial intelligence.

AI literacy and competency matter. Human oversight matters. Organizational guardrails matter. Traceability matters. Continuous quality monitoring matters. Multidisciplinary governance matters.

Most importantly, clinical judgment must remain identifiable and accountable throughout the process.

The field does not need to wait for a significant adverse event to determine what responsible AI governance should look like.

We already know enough about clinical quality, risk management, professional competency, and the history of technological adoption to begin building those structures now.

The opportunity before behavioral health is not simply to adopt increasingly capable AI.

It is to ensure that our systems of governance become equally capable.