Existing law requires, on or before September 1, 2024, the Department of Technology, within the Government Operations Agency, to conduct, in coordination with other interagency bodies as it deems appropriate, a comprehensive inventory of all high-risk automated decision systems that have been proposed for use, development, or procurement by, or are being used, developed, or procured by, any state agency. Existing law requires the department to annually submit a report of that comprehensive inventory to the Assembly Committee on Privacy and Consumer Protection and the Senate Committee on Governmental Organization.

Existing law, the Transparency in Frontier Artificial Intelligence Act, among other things related to ensuring the safety of certain artificial intelligence models, requires a large frontier developer to write, implement, and clearly and conspicuously publish on its internet website a frontier AI framework that applies to the large frontier developer’s frontier models and describes how the large frontier developer approaches, among other things, incorporating national standards, international standards, and industry-consensus best practices into its frontier AI framework.

This bill would require the agency to establish the California AI Standards and Safety Commission and would provide for its membership, as specified.

This bill would require the commission to take certain actions related to the safety of artificial intelligence, including maintaining formal liaison relationships with state agencies deploying or procuring artificial intelligence, providing artificial intelligence technical expertise and artificial intelligence risk assessment, and designating one or more entities as independent verification organizations (IVOs), as specified.

This bill would require an IVO to take certain actions, including to implement the plan for artificial intelligence risk mitigation that it submitted to the commission when it applied for designation as an IVO and submit to the Legislature and to the commission an annual report that addresses, among other things, the adequacy of existing evaluation resources and mitigation measures to mitigate observed and potential risks.

This bill would authorize the commission to establish a fee structure for charging fees to applicants and designated IVOs to offset the reasonable costs incurred by the commission in carrying out its duties pursuant to the bill and adopt regulations necessary to administer the bill.