Automated decision systems.

Assembly Member Bauer-Kahan's comprehensive legislation establishes new oversight requirements for automated decision systems (ADS) that make or facilitate consequential decisions affecting Californians' access to essential services and opportunities. The bill creates a framework of obligations for both developers and deployers of these systems, aiming to ensure transparency and accountability in automated decision-making.

The legislation requires developers to conduct regular performance evaluations of covered ADS systems, assessing their accuracy, reliability, and potential impacts across different populations. These evaluations must examine whether the systems produce disparate outcomes for protected groups and document any measures taken to mitigate such effects. Developers must provide deployers with detailed information about system capabilities, limitations, and proper usage.

Beginning January 1, 2027, organizations deploying covered ADS must notify individuals subject to automated decisions, offering them the opportunity to opt out of automated processing and appeal outcomes. Deployers serving more than 5,999 people must undergo independent third-party audits every three years to assess system performance and impacts. The bill mandates retention of comprehensive documentation about system usage, decisions, and audit results.

The measure authorizes the Attorney General, Civil Rights Department, and other specified entities to bring civil actions for violations, with penalties up to $25,000 per incident. While protecting trade secrets, it requires disclosure of unredacted performance evaluations and impact assessments to enforcement authorities upon request. The legislation exempts certain systems like cybersecurity tools and aircraft operations, and clarifies that use of consumer credit scores alone does not trigger these requirements.

The bill amends California's Unruh Civil Rights Act to address automated decision-making while preserving existing anti-discrimination protections. It establishes that compliance with the new ADS requirements, while relevant, is not conclusive in determining civil rights violations involving automated systems.