ChatGPT and HIPAA Compliance: What Healthcare Professionals Can and Cannot Do
- MLJ CONSULTANCY LLC
- 7 hours ago
- 13 min read
ChatGPT and HIPAA Compliance | A single pasted note can create a compliance problem.
A clinician asks an artificial intelligence tool to “summarize this discharge plan.” A billing team member asks it to “rewrite this appeal letter.” A care coordinator asks it to “make this patient message easier to understand.” Each task sounds routine. The risk appears when the prompt includes a patient’s name, date of birth, diagnosis, medical record number, appointment details, or any other information that could identify the person.
That is where ChatGPT and HIPAA compliance become a practical, daily issue for healthcare organizations.
The Health Insurance Portability and Accountability Act, better known as HIPAA, sets federal rules for protecting certain health information in the United States. It does not ban artificial intelligence. It also does not give healthcare teams permission to place patient information into any tool that seems useful. HIPAA focuses on how protected health information is used, disclosed, stored, accessed, and safeguarded.
This article is informational only and is not legal advice. Healthcare organizations should involve compliance, privacy, security, and legal staff before approving any artificial intelligence tool for patient-related work.
What HIPAA compliance means when artificial intelligence enters the workflow
HIPAA applies to covered entities and business associates.
A covered entity usually means a healthcare provider, health plan, or healthcare clearinghouse that handles protected health information in certain ways. Many physician practices, hospitals, dental offices, mental health clinics, pharmacies, and health plans fall into this category.
A business associate is a person or company that performs certain services for a covered entity and needs access to protected health information to do that work. Common examples include billing vendors, transcription services, claims support vendors, technology service providers, and consultants who handle patient data.
Under HIPAA, protected health information includes health information that can identify a person and relates to that person’s health, care, or payment for care. It can appear in many forms:
A name linked to a diagnosis
A medical record number
A date of birth in a patient message
A photo of a wound
An insurance member number
Appointment details for a specific person
A note that says someone received treatment at a specific clinic
A rare diagnosis combined with location or age
Lab results tied to a patient account
HIPAA does not care only about medical charts. It also applies to emails, chat messages, uploaded files, screenshots, transcripts, recordings, and copied text if they contain protected health information.
When an artificial intelligence tool receives protected health information, the healthcare organization must ask a key question:
Is this tool approved to receive this information under HIPAA, with the right contract and safeguards in place?
If the answer is no, the safer rule is simple. Do not enter protected health information.
Artificial intelligence is not automatically HIPAA compliant
No tool is “HIPAA compliant” just because it uses encryption, has a privacy policy, or says it protects data. HIPAA compliance depends on the full arrangement.
For an artificial intelligence tool to be used with protected health information, a healthcare organization usually needs all of the following:
A signed business associate agreement
Clear limits on how the vendor may use the information
Security controls that protect the data
Access controls that limit who can use the tool
Audit records that show activity when needed
Policies for retention and deletion
Staff training on approved use
A risk review before the tool goes live
A business associate agreement is a written contract required by HIPAA when a vendor handles protected health information for a covered entity. It explains what the vendor may do with the information, how the vendor must protect it, and what happens if there is a security incident.
The U.S. Department of Health and Human Services, through its Office for Civil Rights, enforces HIPAA. Its guidance makes clear that covered entities and business associates must apply the HIPAA Privacy Rule and Security Rule when protected health information is created, received, maintained, or transmitted. In plain language, if patient-identifying health information moves through a system, that system matters.
This is why the “copy and paste” habit is risky. A staff member may see an artificial intelligence chat box as a drafting tool, not as a place where patient data is being sent outside the organization. From a HIPAA view, that distinction may not help. If identifiable patient information leaves an approved system and goes into an unapproved tool, the organization may have created an unauthorized disclosure.
What healthcare professionals can do with artificial intelligence tools
Artificial intelligence tools can be useful in healthcare when used carefully. The safest uses avoid protected health information or occur inside approved systems with the right safeguards.
The examples below focus on uses that do not require entering identifiable patient information.
Draft general patient education material
A clinician can ask for plain-language explanations of general health topics, then review the output for accuracy.
Safe example:
“Write a sixth-grade reading level explanation of what blood pressure is. Do not give personal medical advice. Include a reminder to contact a healthcare professional with questions.”
This type of prompt does not include a patient name, medical history, medication list, or test result. The output still needs clinical review. Artificial intelligence can produce statements that sound confident but are incomplete or wrong.
Good uses include:
Explaining common preventive screenings
Drafting general discharge instruction templates
Creating a handout about how to prepare for a routine visit
Rewriting a clinic policy into easier language
Making a general medication safety checklist
The key word is general. Once the prompt includes facts about a specific patient, the risk changes.
Improve internal templates without patient details
Healthcare teams often use repeated documents. Artificial intelligence can help make these clearer.
Safe example:
“Rewrite this appointment reminder template in a warm but professional tone. Do not add medical advice. Template: Please arrive 15 minutes early and bring your insurance card, photo identification, and medication list.”
That prompt does not reveal anything about a specific person’s diagnosis or care. It is a template improvement task.
Good candidates include:
New patient welcome instructions
Call center scripts
Missed appointment letters without patient details
General consent explanation drafts for review
Staff reminders about documentation standards
A privacy review still helps, especially before staff use any tool regularly.
Create training examples with fictional information
Artificial intelligence can help build role-play scenarios, test questions, or documentation exercises if the facts are fictional.
Safe example:
“Create a fictional training scenario for a front desk employee who needs to verify identity before discussing appointment information. Use made-up names and made-up dates.”
This can support privacy training without exposing real patients.
The safer approach is to make the prompt clearly fictional. Do not use “similar to a real patient” details. Small combinations of facts can identify someone, especially in smaller communities or specialty practices.
Summarize public health information
Artificial intelligence tools can help summarize publicly available information, such as general guidance from public health agencies or professional organizations. A staff member might ask for a short summary of general vaccination guidance or a plain-language version of a public notice.
The output should still be checked against the original source. Medical guidance changes, and artificial intelligence tools may not reflect the most current recommendations.
A safer prompt might say:
“Summarize this public guidance in plain English for adult patients. Do not add recommendations that are not in the text.”
Then paste only public text, not patient-specific details.
What healthcare professionals should not do
The highest-risk uses involve entering patient-identifying information into an artificial intelligence tool that has not been approved for protected health information.
A practical rule works well:
If the information could identify a patient, do not put it into an unapproved tool.
That includes obvious identifiers, such as name and medical record number. It also includes combinations of facts that could point to one person.
Do not paste patient notes into an unapproved tool
Unsafe example:
“Summarize this visit note for Mary Smith, date of birth 4/12/1958, who was seen today for chest pain and diabetes follow-up.”
This includes a name, date of birth, diagnoses, and visit information. If the tool is not approved under HIPAA, this is not an acceptable use.
Even removing the name may not be enough.
Still risky:
“Summarize this note for a 42-year-old teacher in a small town who is 22 weeks pregnant and was treated for a rare blood disorder.”
The patient might be identifiable from the combination of age, location, pregnancy status, occupation, timing, and diagnosis.
Do not upload lab reports, images, or messages with identifiers
Files often contain hidden or visible identifiers. A lab report may include a patient name, account number, accession number, date, ordering provider, facility, and timestamp. A clinical photo may include a face, tattoo, bracelet, room label, or metadata created by the device. A patient portal message may include names, dates, medication details, and contact information.
Unsafe uses include uploading:
Discharge summaries
Lab reports
Referral letters
Insurance documents
Prior authorization forms
Patient portal messages
Photos of injuries or skin findings
Audio files from patient calls
Screenshots from the medical record
If a tool is not approved to handle protected health information, uploading documents can create more risk than typing a short prompt.
Do not ask for patient-specific diagnosis or treatment decisions
Artificial intelligence should not replace professional judgment. Even if a prompt contains no identifiers, asking a general tool to diagnose or recommend treatment for a real patient raises quality, safety, and liability concerns.
Unsafe example:
“This patient has severe abdominal pain, fever, and these lab values. What is the diagnosis and what medication should I prescribe?”
That prompt may contain protected health information if the patient can be identified. It also asks the tool to guide care in a way that may be unsafe without full clinical context.
A safer educational prompt removes the patient connection:
“List common causes of abdominal pain and fever in adults for educational review. Include a reminder that diagnosis requires clinical evaluation.”
Even then, the response must be checked by a qualified clinician.
Do not rely on artificial intelligence to remove identifiers unless approved
Some staff may try to paste a full note into a tool and ask it to “remove all patient information.” That approach defeats the purpose. The unapproved tool already received the protected health information before it could remove anything.
A safer process removes identifiers before any prompt is entered. When possible, use an approved internal method for de-identification. If the details are not needed, do not include them.
Do not assume staff accounts are private enough for patient data
A login screen, password, or paid account does not automatically make a tool safe for protected health information. Staff accounts may not include a business associate agreement, enterprise controls, audit records, or the settings required by an organization’s privacy policy.
Before using any artificial intelligence tool with patient data, confirm approval through the organization’s normal review process. Verbal assurance is not enough. HIPAA compliance depends on documented safeguards and responsibilities.
A practical can-and-cannot guide
The table below gives common healthcare tasks and a safer way to handle each one.
Task | Safer use | Do not do this |
Rewrite a patient education handout | Use general text with no patient details, then have a clinician review it | Paste a patient’s discharge instructions with name, diagnosis, and procedure date |
Create a phone script | Ask for a general script about identity verification or appointment reminders | Include real appointment details for a named patient |
Summarize a policy | Paste an internal policy that contains no patient information, if allowed by policy | Paste incident reports or patient complaints with identifying details |
Build training scenarios | Use fictional names, fictional dates, and made-up facts | Base the scenario on a recent patient case that staff may recognize |
Draft an appeal letter | Use a blank template and general wording | Enter a member number, diagnosis, provider name, service dates, and claim details |
Explain a diagnosis | Ask for a general explanation for clinician review | Ask the tool to explain a real patient’s diagnosis using their chart note |
Improve documentation quality | Ask for a checklist of what a good note often includes | Paste real clinical notes into an unapproved tool |
This table does not replace a privacy review. It gives a working standard for everyday judgment.
Why de-identification is harder than it sounds
Many people think de-identification means removing the patient’s name. HIPAA is broader than that.
Under HIPAA, information can remain identifiable if it includes obvious identifiers or enough detail to recognize the person. Common identifiers include names, dates related to care, telephone numbers, addresses, email addresses, Social Security numbers, medical record numbers, account numbers, photos of the face, and similar details.
There are also less obvious clues. A rare disease, small town, unusual injury, exact age for an older adult, or specific date of service can narrow the field quickly.
For example, this prompt may look anonymous:
“Rewrite this note for a 91-year-old retired judge in rural Kansas who had a fall during a church event last Friday and was transferred by helicopter.”
There is no name. Even so, the combination of age, occupation, location, event, date, and transport method may identify the person.
A safer version would be:
“Rewrite this fictional fall prevention note in plain language for older adults. Do not include personal details.”
When in doubt, use fictional details or ask for a general framework.
Safer alternatives for managing sensitive patient data
Healthcare teams do not need to avoid all artificial intelligence or automation. They need approved tools, clear rules, and safer workflows.
Use approved systems designed for protected health information
The safest place for patient data is the set of systems your organization has already approved. This may include the medical record system, secure messaging tools, billing systems, document management systems, dictation tools, or analytics platforms that have gone through privacy and security review.
Before any new artificial intelligence tool touches patient data, confirm that it has:
A signed business associate agreement when required
Written approval from privacy and security leadership
User access controls
Audit logs
Clear data retention terms
A process for handling security incidents
Staff training materials
A defined purpose for use
If the tool cannot meet those requirements, use it only with non-identifiable or fictional information.
Create an internal prompt policy
A short policy can prevent many problems. It should explain what staff may and may not enter into artificial intelligence tools.
A useful policy should answer:
Which tools are approved
Who may use them
Whether protected health information is allowed
What types of tasks are permitted
What information must never be entered
How outputs must be reviewed
Who to contact with questions
What to do if someone enters patient information by mistake
The policy should use plain examples. Staff should not need to interpret legal language during a busy clinic day.
For example:
Allowed:
“Create a general checklist for preparing for a colonoscopy, subject to clinical review.”
Not allowed:
“Create instructions for John Doe’s colonoscopy on May 6, including his medication list and medical history.”
That level of detail makes the rule easier to apply.
Build approved templates for common tasks
Many risky prompts happen because staff are trying to save time. Templates can help them do that safely.
Good templates include:
General discharge instruction frameworks
Appointment reminder language
Prior authorization outline language without patient details
Patient education reading-level prompts
Staff privacy training scenarios
Call scripts for identity verification
Plain-language explanation formats
The best templates include warnings near the prompt field, such as:
Do not include patient names, dates of birth, medical record numbers, claim numbers, photos, lab reports, visit notes, or any information that could identify a patient.
A simple reminder at the moment of use is often more effective than a long policy stored elsewhere.
Use minimum necessary information
HIPAA includes a “minimum necessary” principle for many uses and disclosures. In simple terms, do not use or share more protected health information than needed for the task.
That principle also helps with artificial intelligence. If a task does not require patient details, leave them out. If a general example works, use that instead. If the same result can be achieved with fake facts, choose fake facts.
For example, a staff educator does not need a real patient story to teach privacy awareness. A fictional case can serve the same purpose.
Separate clinical judgment from drafting help
Artificial intelligence may help draft, summarize, or reword, but healthcare professionals remain responsible for the final content.
Before using any output, check for:
Medical accuracy
Missing warnings
Incorrect dosing or timing
Overly broad advice
Reading level
Scope of practice concerns
Organization policy requirements
Required disclaimers or follow-up instructions
Artificial intelligence can produce false or misleading information. It can also invent details that were not in the prompt. For clinical content, human review is not optional.
What to do if protected health information was entered by mistake
Mistakes happen. The response should be prompt and documented.
A practical response plan includes these steps:
Stop using the tool for that task
Do not continue the conversation or add more details.
Record what happened
Note the tool used, date, time, user, prompt content if available, and type of information entered.
Notify the appropriate internal team
Follow the organization’s privacy or security incident reporting process.
Preserve relevant records
Do not delete evidence unless instructed by the proper internal team. The organization may need to assess what occurred.
Review whether a breach analysis is required
HIPAA has specific breach notification rules. The privacy officer or legal team should guide this review.
Update training or controls
If the mistake happened because the rules were unclear, fix the process.
Avoid blaming staff before the facts are clear. Many incidents come from unclear policies, rushed workflows, or tools that were easy to access without proper warnings.
A simple approval checklist before using artificial intelligence with patient data
Before protected health information goes into any artificial intelligence system, the organization should be able to answer yes to each question below.
Has privacy leadership approved this use?
Has security leadership reviewed the tool?
Is there a signed business associate agreement if HIPAA requires one?
Does the agreement limit how the vendor may use the data?
Are access controls in place?
Are audit logs available?
Is data retention clearly defined?
Can users delete or manage data according to policy?
Are staff trained on what is allowed?
Is there a written process for reporting mistakes?
Has clinical leadership approved any patient-facing or clinical use?
Is human review required before content is used?
If the answer is no or unknown, do not use the tool with patient information.
For organizations that need help reviewing risk, creating policies, or training staff, review the available healthcare compliance support options.
Frequently asked questions
Is it ever okay to use ChatGPT with patient information?
Only if the organization has approved that specific use, the required safeguards are in place, and a business associate agreement exists when HIPAA requires one. Without that approval, do not enter protected health information.
Can staff use artificial intelligence to write general patient education materials?
Yes, if no protected health information is entered and the content is reviewed by a qualified person before use. General education drafts are lower risk than patient-specific prompts.
Is removing a patient’s name enough?
No. Dates, locations, phone numbers, medical record numbers, rare conditions, photos, and combinations of details can still identify a patient. De-identification requires more than deleting a name.
What should a clinic do first before allowing artificial intelligence tools?
Start with a written policy, an approved tool list, staff training, and a review process for any tool that may handle protected health information.
Who enforces HIPAA?
The Office for Civil Rights within the U.S. Department of Health and Human Services enforces HIPAA privacy and security rules.
The practical takeaway
Artificial intelligence can help healthcare teams write clearer materials, build training content, and reduce repetitive drafting work. The risk begins when patient-identifying information enters a tool that has not been approved to receive it.
The safest default is clear:
Use artificial intelligence for general, fictional, or properly approved tasks. Keep protected health information out of unapproved tools.
For healthcare professionals, that rule is easy to remember and hard to overstate. A helpful prompt should never come at the cost of patient privacy.
Comments