Artificial intelligence is rapidly moving from concept to practical application in pathology and laboratory medicine, with emerging use cases ranging from digital pathology algorithms and diagnostic decision support to workflow automation, quality monitoring, and administrative efficiency. As adoption accelerates, pathology practices and laboratories face growing questions regarding compliance oversight, data privacy, vendor accountability, professional liability, and responsible implementation.

During a recent meeting of PNPL’s Compliance Officers Networking Team, Eric Fish, Partner at Hooper, Lundy & Bookman and a nationally recognized expert in digital health regulation, examined the legal, ethical, and governance challenges associated with AI-enabled healthcare technologies. While many organizations are focused on the potential benefits of AI, Fish emphasized that pathology leaders must also establish appropriate safeguards, accountability structures, and oversight processes to ensure these tools are deployed responsibly and in compliance with evolving regulatory expectations.

Understanding AI Beyond the Hype

Fish began by clarifying the distinctions between artificial intelligence, traditional machine learning, and generative AI. While these technologies are often grouped together, each serves different purposes and presents unique opportunities and risks. He emphasized that before implementing any AI solution, pathology practices and laboratories should clearly identify the problem they are attempting to solve. Whether the objective is improving diagnostic accuracy, reducing administrative burden, enhancing decision support, streamlining workflows, or increasing productivity, success depends on defining measurable outcomes and determining whether AI is truly the right tool for the task.

The discussion highlighted the growing presence of AI throughout healthcare and laboratory medicine, including diagnostic imaging, digital pathology, clinical decision support, virtual assistants, and administrative functions. As utilization expands, compliance professionals are increasingly becoming key stakeholders in decisions that were once viewed primarily as technology initiatives.

The Growing Risk of “Shadow AI”

One of the most significant concerns raised during the session was the emergence of “shadow AI”—the use of unauthorized AI applications by clinicians and staff outside approved review processes. Fish noted that many healthcare professionals are already experimenting with publicly available tools to summarize records, draft communications, analyze information, and support clinical tasks without formal approval.

For pathology practices and laboratories, such activity can create substantial exposure because it bypasses established security measures, privacy safeguards, and internal review mechanisms. Potential consequences include unauthorized disclosure of protected health information, patient safety concerns, regulatory violations, and increased institutional liability. The discussion underscored that oversight efforts cannot focus solely on approved platforms; they must also address informal use occurring throughout the organization.

Oversight Is Struggling to Keep Pace

A central theme of the presentation was the widening gap between implementation and organizational readiness. Fish cited findings showing that while AI use has become nearly universal among health systems, only a small percentage have developed mature oversight structures. Many remain in the policy-development stage even as deployment accelerates. Additional concerns include limited audit readiness, insufficient evaluation of bias and performance drift, and a lack of transparency regarding how vendors develop, validate, and maintain their models.

This imbalance creates a situation in which healthcare providers are increasingly relying on technologies that may not yet be supported by adequate accountability mechanisms. For pathology leaders evaluating digital pathology algorithms, image analysis tools, or AI-assisted workflow solutions, the challenge extends beyond approving a product—it requires building a sustainable framework capable of managing risk throughout the lifecycle of the technology.

Building a Strong Accountability Framework

Fish emphasized that no single model works for every organization. Instead, structures should be tailored according to risk and organizational needs. Key components include formal policies governing selection and use, clearly defined responsibilities, and designated leaders responsible for coordinating oversight across departments. He stressed the importance of avoiding silos and ensuring that clinical, technical, legal, compliance, and patient perspectives are represented.

Transparency emerged as another critical principle. Staff and patients should understand how AI is being used, what role it plays in decision-making, and what safeguards exist to protect privacy and quality. Fish also emphasized the need for continuous evaluation to assess effectiveness, identify bias, and detect performance degradation over time. Accountability, he noted, is not a one-time exercise but a continuing discipline that must evolve alongside the technology.

Vendor Contracts Require Greater Scrutiny

The session devoted considerable attention to vendor contracting, an area that often receives less attention than technology evaluation but can have significant long-term implications. Fish cautioned that many AI vendors rely on contract language that shifts substantial responsibility to healthcare providers while limiting the vendor’s own exposure. Compliance and legal teams should carefully review business associate agreements, data-use provisions, indemnification clauses, disclaimers, and liability limitations before moving forward with an AI solution.

Particular attention should be paid to terms such as “de-identified data” and “product improvement.” These seemingly routine provisions can influence how patient information is used beyond the original clinical purpose. Fish encouraged organizations to negotiate meaningful performance expectations, reporting requirements, and clear accountability measures regarding accuracy and maintenance.

For pathology practices considering digital pathology platforms or AI-enabled diagnostic tools, vendor due diligence should extend beyond technical capabilities to include contractual protections, transparency commitments, and ongoing performance obligations.

Model Cards and Responsible AI Evaluation

Fish introduced the concept of “model cards,” an emerging tool designed to improve transparency and accountability. Developed by organizations such as CHAI and the Health AI Partnership, model cards provide structured documentation describing an AI model’s intended use, performance metrics, limitations, validation methods, and known risks.

These resources can help pathology groups and laboratories determine whether a tool is appropriate for a particular clinical setting, identify potential sources of bias, support regulatory obligations, and guide future reassessment efforts. Participants discussed how model cards may become increasingly important as healthcare organizations seek more standardized approaches to evaluating AI technologies.

Preparing for a Rapidly Changing Regulatory Environment

The presentation also examined the accelerating pace of AI-related regulation at both the federal and state levels. Fish reviewed emerging federal initiatives, including evolving FDA oversight, interoperability requirements, and broader national policy discussions concerning AI risk management. He also highlighted extensive legislative activity across the states addressing issues such as algorithmic bias, transparency requirements, insurance determinations, and human oversight of automated systems.

The key message was that legal compliance and AI management are becoming increasingly interconnected. Pathology practices and laboratories that establish thoughtful oversight processes today will be better positioned to adapt as requirements continue to evolve.

Professional Responsibility and Medical Liability

The discussion concluded with an examination of professionalism and malpractice considerations associated with AI-assisted decision-making. Fish reviewed guidance from state medical boards emphasizing that clinicians retain responsibility for patient care decisions even when AI tools are involved. Technology may assist decision-making, but it does not replace professional judgment. Physicians must be prepared to explain why they relied on a recommendation, why they chose to deviate from one, and how the technology informed their overall assessment.

The presentation also explored emerging liability questions. As AI becomes more integrated into healthcare delivery, courts and juries may increasingly evaluate whether providers appropriately used—or failed to use—available tools. Comprehensive documentation therefore remains essential. Pathologists and laboratory leaders should document why a solution was selected, how outputs were evaluated, and how recommendations contributed to final decisions. Clear communication regarding the role of AI in patient care may also become an important component of risk management.

Key Takeaways

Fish closed the session with five practical recommendations for organizations seeking to strengthen their approach to AI:

1.      Conduct thorough vendor due diligence and independently validate performance claims.

2.      Strengthen contractual protections related to data use, accountability, and liability.

3.      Establish regular review processes to assess accuracy and detect model degradation.

4.      Maintain meaningful clinical oversight and prevent AI-generated content from entering records without appropriate review.

5.      Prioritize transparency by informing patients and staff how AI is being used and obtaining appropriate consent when necessary.

Conclusion

As artificial intelligence becomes more deeply integrated into pathology and laboratory medicine, leaders must balance innovation with accountability. Fish’s presentation emphasized that success requires far more than selecting the right technology. Thoughtful policies, rigorous vendor review, continuous performance assessment, regulatory awareness, and a steadfast commitment to professional judgment will be essential for organizations seeking to realize the benefits of AI while managing its risks. For pathology practices and laboratories, the message was clear: accountability structures must advance at the same pace as the technology itself.

Panel of National Pathology Leaders’ Member Practices have access to a wealth of information and tools to help navigate management challenges, including exclusive access to reports, data, and surveys.