Using AI Responsibly in Healthcare and Human Services

RESPONSIBLE INNOVATION · 9 MIN READ

Practical principles for privacy, human oversight, transparency, appropriate use, and accountable decision-making.

IN THIS ARTICLE

Responsibility comes first

Six practical principles

Build governance into work

A responsible starting point

Artificial intelligence can help people find information, organize work, identify patterns, and reduce repetitive administrative effort. In healthcare and human services, however, usefulness is not enough. AI must be introduced in ways that protect people, preserve professional judgment, and make responsibility visible.

This article presents a practical governance framework, not legal, clinical, privacy, or regulatory advice. Requirements depend on the organization, data, technology, and intended use.

Responsibility comes before automation

The most important question is not whether an AI tool can perform a task. It is whether the task should be supported by AI, under what conditions, and with what safeguards.

A low-risk use—such as drafting a meeting agenda from non-sensitive notes—requires different controls than a tool that summarizes a clinical record, recommends a service, identifies a person as high risk, or influences access to benefits. Organizations should evaluate the consequences of error before selecting the technology.

The World Health Organization emphasizes human autonomy, safety, transparency, accountability, inclusion, and sustainability in AI for health. The NIST AI Risk Management Framework similarly encourages organizations to address validity, safety, security, transparency, explainability, privacy, and harmful bias throughout the AI lifecycle.

A USEFUL RULE

The greater the potential effect on a person's health, rights, services, or opportunities, the stronger the review, evidence, oversight, and appeal mechanisms should be.

Six practical principles

01 Define an appropriate use before choosing a tool

Begin with a specific problem and a clearly bounded use case. Identify who will use the output, what decision it may influence, what good performance looks like, and what happens if the output is wrong.

Avoid open-ended goals such as “use AI to improve care.” A more responsible starting point might be “help authorized staff categorize non-clinical resource requests, with every result reviewed before it affects workflow.”

02 Protect privacy throughout the information flow

Teams should know what information enters an AI system, where it is processed, how long it is retained, whether it may be used to improve a vendor's model, who can access it, and how it can be deleted.

Do not assume a tool is appropriate for protected health information because it is described as secure or enterprise-ready. Covered entities and business associates should evaluate their own obligations, contracts, safeguards, permitted uses, and vendor relationships. HHS guidance also makes clear that de-identification has defined standards; simply removing a name does not necessarily make health information de-identified.

03 Keep meaningful human oversight

Human review should be more than a person clicking “approve.” Reviewers need enough information, authority, time, and training to question an AI output. They should know the system's intended use, limitations, common failure modes, and when escalation is required.

High-impact decisions should not become unreviewed automation. People affected by an AI-supported process should have an understandable way to raise concerns, correct information, and request human reconsideration.

04 Make the use of AI transparent

Transparency should be appropriate to the audience. Staff may need operating instructions, data-flow documentation, testing results, and escalation procedures. Leadership may need risk, performance, and incident reporting. People receiving services may need a clear explanation of when AI is involved and how it affects them.

Transparency does not require exposing proprietary source code. It does require enough information to understand the system's purpose, limits, inputs, outputs, responsible owner, and available recourse.

05 Test for performance, equity, and context

A model that performs well in one population, program, or setting may not perform the same way elsewhere. Organizations should test the actual use case with representative data and examine whether errors or outcomes differ across relevant groups.

Evaluation should continue after launch. Data changes, workflows evolve, vendors update models, and staff may use tools in ways that were not anticipated. Monitoring should include performance, overrides, complaints, incidents, and unexpected patterns.

06 Assign accountability that cannot be delegated

A vendor may provide the technology, but the organization remains responsible for deciding whether and how it is used. Every AI use case should have a named owner, an approval path, documented controls, and a process for suspending use when safety or performance is uncertain.

Procurement, privacy, security, legal, clinical or program leadership, quality improvement, frontline staff, and people with lived experience may all contribute important perspectives. The level of review should match the level of risk.

Build governance into everyday work

Responsible AI should not depend on individual caution alone. It should be supported by repeatable organizational practices.

Before use

Define the use case and prohibited uses

Assess impact, privacy, security, and equity risks

Review vendors, contracts, and information flows

Test with representative scenarios and users

During use

Keep human review proportionate to risk

Document outputs, overrides, and decisions

Monitor performance, incidents, and complaints

Reassess after material system or workflow changes

A simple AI inventory can be a powerful first control. Record each tool, owner, purpose, users, data involved, vendor, approval status, risk level, review date, and monitoring plan. This gives leadership a clear view of where AI is already being used—including unofficial or experimental use.

A responsible starting point

Organizations do not need to solve every AI governance question before learning. They do need boundaries. Start with low-risk, well-defined uses; exclude sensitive data unless the environment and agreements are appropriate; require meaningful review; document lessons; and expand only when evidence supports doing so.

Responsible AI is not a one-time approval. It is an ongoing practice of matching technology to purpose, evaluating consequences, listening to affected people, and changing course when the evidence requires it.

Authoritative guidance

This article was informed by the following public guidance:

World Health Organization: Ethics and governance of artificial intelligence for health

NIST AI Risk Management Framework

U.S. HHS: Guidance on de-identification of protected health information

INNOVATION WITH ACCOUNTABILITY

AI should strengthen human judgment—not hide responsibility.

Elevita helps healthcare and human-services organizations design practical technology and governance around their work, information, and accountability requirements.

Contact Elevita →