From "Too Risky to Use" to "Usable Because the Conditions Can Be Verified"

The Controls Required by Public Guidelines and the “Implementation Gap” Filled by Privacy Assurance

In healthcare settings, the scenarios for utilizing AI are expanding rapidly, including summarizing medical records, assisting in discharge summary creation, classifying inquiries, and reviewing documents.

However, when processing patient information with external AI services, is an explanation such as “we deleted the names” or “we removed the patient IDs” sufficient to move forward with formal operational use?

We believe that alone is not enough. What is required is not merely the fact that patient information was processed. It must be proven that only verified data was actually transmitted to the designated destination under predetermined conditions. Furthermore, the outputs returned by the AI must be adopted into formal healthcare operations only if they pass verification. This entire sequence of processing must be verifiable in an auditable manner after the fact.

▼Cyber Assurance Press Release(JP)https://prtimes.jp/main/html/rd/p/000000003.000182721.html

1. De-identification and Actual Transmission Are Separate Processes

For example, suppose a healthcare institution removes names and patient IDs from clinical records and transmits them to an external AI to generate summaries. Even if the pre-transmission data was appropriately processed, what if information like email addresses is added during the subsequent communication process (e.g., internal processing by SDKs or communication libraries)? There remains a risk that content differing from the verified data will be passed to the actual transmission process.

Additionally, settings for the transmission destination or AI service might be altered, resulting in data being sent to an unintended environment. A one-time transmission permission granted for one instance could also be reused to send different patient data. In such cases, while a record stating that “de-identification was performed” exists, it cannot prove that the de-identified content was the exact payload actually transmitted.

De-identification is an operation performed on data. What is necessary, however, is a mechanism that links those processing results with the “actual external transmission” through auditable evidence after the fact.

2. Protecting Only the Outbound Boundary Is Insufficient for Medical AI

Another easily overlooked aspect is the handling of “outputs” returned by AI.

Even if outbound information is processed appropriately, there is a possibility that patient-identifying information might reappear (re-emerge) within the AI’s output. Furthermore, AI-generated text may contain factual inaccuracies, unintended free-text entries, or formats incompatible with operational systems. If such outputs are registered in electronic health records (EHR) or patient management systems without inspection, AI-generated candidates become the official records of the medical institution as-is.

Therefore, medical AI requires the management of two distinct boundaries:

1. The boundary where information leaves the healthcare institution for the AI

2. The boundary where AI results return into the formal workflows of the healthcare institution

To establish privacy protection as a formal operational reality, boundary management at both ends is indispensable.

3. Requirements Set Out in Public Guidelines

This awareness is not merely theoretical. Management requirements are explicitly stated in guidelines and directives published by government agencies and international organizations.

① Personal Information Protection Commission (PPC): “Cautionary Note on the Use of Generative AI Services” [1]

The PPC’s cautionary note advises organizations that, when entering prompts containing personal information, they should thoroughly confirm in advance that the input remains within the scope necessary to achieve the intended purpose of use. Furthermore, where personal data is entered without the individual’s consent, the notice advises organizations to sufficiently confirm in advance that the AI service provider will not use the data for machine learning or other purposes beyond generating the requested response.

② Ministry of Health, Labour and Welfare (MHLW): “Safety Management Guidelines for Medical Information Systems” Version 7.0 [2]

Version 7.0 of the MHLW guidelines specifies that network services capable of transferring medical information outside the institution must be restricted exclusively to those approved by the planning manager, with connection controls implemented as necessary. It also calls for countermeasures against leaks caused by network service misconfigurations or operational errors, alongside periodic reviews of data export statuses. Furthermore, regarding the disposal of medical information entrusted to external entities, it specifies requiring evidence from contractors that can verify appropriate destruction. The guideline highlights an important principle: even when proving destruction on cloud services is difficult, evidence regarding destruction procedures and the actual processing performed must be required.

③ Requirements Regarding Pseudonymized Information, Entrustment, and Safety Management [3], [4]

Guidelines such as the MHLW’s “Guidelines on the Utilization of Medical Digital Data for AI Research and Development” [3] clarify that even if medical information is pseudonymized, it can still constitute personal information on the healthcare provider side if it can be easily cross-referenced with original records. The guidelines also require the authority and responsibilities of personnel handling removed information and information about the processing methods to be clearly defined, together with appropriate management measures. Additionally, as indicated in the “Guidance on the Proper Handling of Personal Information by Medical and Long-Term Care Service Providers” [4], when external AI usage constitutes the outsourcing of personal data processing, medical institutions are required to exercise necessary and appropriate supervision over contractors. To make this requirement effective, it becomes crucial to have mechanisms allowing medical institutions to verify not just terms of service, but actual transmission conditions and processing statuses on their side.

④ International Guidance on AI Output Risks and Post-Deployment Audits [5]

The World Health Organization (WHO) ethics and governance guidance on Large Multi-modal Models (LMMs) [5] highlights risks such as inaccurate, biased output, automation bias, and cyber risks. It recommends implementing independent post-deployment audits and impact evaluations, including data protection, during large-scale implementations.

4. Technical Means Filling the “Implementation Gap” Left by Guidelines

While official guidelines outline these management requirements, they do not specify the exact technical methods for binding verified data, configurations, and destinations to each individual transmission.

Guidelines do not prescribe specific methods to verify whether the agreements confirmed in contracts or screen interfaces are strictly maintained during “each individual communication transaction” and to leave audit-ready proof. Here lies the “implementation gap” connecting administrative management requirements to actual transmission execution.

The privacy assurance technology for which we filed a patent application bridges this implementation gap through the following process:

1. Alignment Between Verified Content and Actual Transmission: Moving beyond static verification of processed data, the system seals and fixes the payload and settings of the final application-layer request, then extracts and verifies the actual disclosure elements against the approved conditions.

2. Single-Use Transmission Authorization: Single-use authorization is granted on the strict condition that verified criteria completely match actual transmission criteria, rejecting any re-use of the same authorization or execution of unauthorized transmission requests.

3. Independent Management of Outcome-Unknown States: When transmission success is uncertain due to communication failures, reservations are not casually released via simple timeouts; rather, they are kept pending until reliable evidence is gathered, preventing the loss of accountability.

4. Formal Adoption Gate for AI Outputs: Outputs from external AI are independently verified for risks like re-exposure of personal information or unresolved verification duties. Only upon passing inspection is writing (state modification) to formal operational systems, such as EHRs, permitted.

【Technical Overview: Summary of Patent Structure】 In this technology, the final application-layer transmission request is sealed immediately before it is handed to the TLS-protected transport layer. The system then extracts and verifies the actual disclosure information and determines the resulting disclosure amount. By atomically coupling this disclosure volume with the transmission authorization and privacy budget reservation as an “atomic conditional update,” duplicate consumption and unauthorized diversion are prevented.

5. Separating Privacy Assurance from AI’s Medical Accuracy

Privacy assurance is not a technology designed to guarantee the medical validity or clinical accuracy of diagnostic candidates or treatment suggestions generated by AI. Medical judgments require separate validation, including underlying diagnostic test results, medical images, observation timestamps, clinical conditions, and physician review.

However, no matter how clinically superior an AI may be, if patient data exports and output adoption cannot be properly managed at the system level, healthcare institutions will find it difficult to adopt it for formal operational use.

“AI Performance” and “Safely Connecting AI to Clinical Workflows” are distinct problems. To successfully achieve the real-world deployment of medical AI, both must be accomplished simultaneously.

6. “Responsibility OS” to Keep Medical AI Moving Forward

Privacy protection is often perceived as a “restriction” designed to prevent information from leaving the organization. However, our goal is not to halt the adoption of AI.

• Which data can be sent, under what conditions, and to which AI?

• Under what conditions can returned results be formally adopted into the institution’s official records?

Defining these criteria in advance, verifying them at runtime, and proceeding only when conditions are fully satisfied — this approach enables moving from “rejecting AI because of risk” to “using AI because the required conditions can be shown to have been met.”

Establishing the conditions under which patient data may be handled, and being able to prove that those conditions were met in every single transaction.

The privacy assurance technology introduced here extends and operationalizes the core principle established in ADIC Cyber Assurance — “no critical execution without a legitimate evidence chain” — into the domain of privacy, specifically governing external patient data transmission and formal AI output adoption.

We have filed a patent application for this privacy assurance technology (Japanese Patent Application №2026–174858).

This technology represents one implementation of a “Responsibility OS” (Responsibility Operating System) that makes patient-data transmission and AI-output adoption verifiable as part of the conditions required for formal operational use in healthcare institutions.

References

[1] Personal Information Protection Commission. Cautionary Note on the Use of Generative AI Services. June 2023.

[2] Ministry of Health, Labour and Welfare. Safety Management Guidelines for Medical Information Systems, Version 7.0. June 2026.

[3] Ministry of Health, Labour and Welfare. Guidelines on the Utilization of Medical Digital Data for AI Research and Development. March 31, 2024.

[4] Personal Information Protection Commission & Ministry of Health, Labour and Welfare. Guidance on the Proper Handling of Personal Information by Medical and Long-Term Care Service Providers. Revised April 2026.

[5] World Health Organization. Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models. WHO Guidance, 2024.