Implications of California Legislation Regulating Generative Artificial Intelligence in Health Care Services

An Overview of Generative AI in Healthcare

According to a survey conducted by McKinsey surveying 100 healthcare leaders which included C-level executives from healthcare services and technology groups, more than 70 percent of respondents claim that they are pursuing or have already implemented generative artificial intelligence (AI) capabilities in the industry. Although there is an increased interest in the capabilities of generative AI, many people are still concerned about the proper ways to regulate its use in healthcare services. When new technology emerges in the healthcare sector, the Food and Drug Administration (FDA) reviews it and requires the producers to demonstrate that the technology is effective at performing certain functions. However, generative AI operates with the help of large language models (LLMs), which means that the AI is able to learn from new datasets and continuously evolve. 

Therefore, one way of regulating generative AI could be to require LLMs to undergo training that physicians and clinicians have undergone. Then, LLMs could be exposed to more training materials and then supervised by trained healthcare professionals. This process of exposing new material to LLMs, supervising and applying their capabilities in controlled environments, and then retesting to ensure LLMs are up to date reduces the potential errors that generative AI could make. Some other ways to reduce potential errors could be providing updated medical data for training or supervision from medical professionals. These issues surrounding generative AI have prompted some states to pass new legislation protecting individuals’ privacy from generative AI. 

California Legislations Regulating Generative AI in Healthcare

Following Utah’s Artificial Intelligence Policy Act, passed in April of 2024, which set regulations on generative AI use in healthcare, California soon followed to address the issue of generative AI in healthcare through three key pieces of legislation that were passed in September of this year. The first legislation is SB 1223, which amended the California Consumer Privacy Act of 2018 to include neural data as sensitive personal information limiting the collection and use of neural data by companies. This legislation acknowledges the potentially invasive nature of generative AI in collecting medical data related to the brain. 

The next legislation is SB 1120, which limits the degree to which health insurers can use AI to determine medical necessity for member health care services. This addresses the increasing use of generative AI to make medical decisions for patients. As generative AI becomes trained on more data, it is able to make more accurate assessments of certain patient’s health. But, this legislation seeks to maintain a balance between utilizing generative AI and human medical professionals in making informed medical decisions. 

The last and potentially most significant legislation is AB 3030, which requires healthcare providers to disclose when they have used generative AI when communicating with their patients. This legislation attempts to enhance transparency between patients and healthcare providers by explicitly stating when generative AI has been used. The use of generative AI alleviates the burden placed on healthcare workers, especially with smaller tasks, such as communicating with patients. 

Implications of Legislations Regulating Generative AI in Healthcare

California’s efforts to regulate generative AI highlight many of the potential benefits it can bring. For example, generative AI chatbots can handle administrative work, such as booking appointments or reminding patients about their appointments. Allowing generative AI to take over these tasks creates more time for human healthcare workers to work with patients one-on-one. 

Furthermore, generative AI can effectively collect information regarding patients’ medical history to keep track of their health. This can be helpful as generative AI chatbots can utilize this information to come up with questions for patients just like how real human professionals inquire about patients. 

However, legislation regulating generative AI must allow for swift development while maintaining control through governance. Different stakeholders must work together to ensure that generative AI can progress while being checked. Designers, engineers, governors, and users are all responsible for the deployment of generative AI in the healthcare industry. Designers help promote generative AI and its features which means they must be educated about the potential risks of failures. Engineers are responsible for the LLMs that enable generative AI chatbots to produce accurate answers. Governors are responsible for coming up with frameworks that allow for the safe deployment of generative AI while following guidelines around data privacy. Finally, users are the people who are affected by generative AI which makes it even more important that they know the risks of implementing generative AI in healthcare.

Future of Generative AI in Healthcare

As generative AI continues to gain interest from many in the healthcare field, it is important that the law takes into consideration concerns that many users may have. It is significant to ensure that human clinicians oversee the implementation of generative AI as they are still responsible for potential errors. Through the continued development of new frameworks, generative AI can be implemented on the national stage to benefit millions of people.

Image Source: Orrick