S T A T E   O F   N E W   Y O R K
 ________________________________________________________________________
 
                                   10625
 
                           I N  A S S E M B L Y
 
                               June 20, 2024
                                ___________
 
 Introduced   by   COMMITTEE  ON  RULES  --  (at  request  of  M.  of  A.
   Bichotte Hermelyn) -- read once and referred to the Committee on Codes
 
 AN ACT to amend the executive law, in relation to the regulation of  the
   use  of  artificial  intelligence and facial recognition technology in
   criminal investigations; and to amend the criminal procedure  law,  in
   relation  to  limiting  the  use  of artificial intelligence-generated
   outputs in court
 
   THE PEOPLE OF THE STATE OF NEW YORK, REPRESENTED IN SENATE AND  ASSEM-
 BLY, DO ENACT AS FOLLOWS:
 
   Section  1.  Legislative  intent.   Artificial intelligence tools like
 Cybercheck and facial recognition technologies have been used extensive-
 ly in criminal investigations across the U.S.,  including  nearly  8,000
 cases  in  40  states.  However,  their use has raised significant legal
 challenges due to the lack of transparency in  their  methodologies  and
 concerns  over accuracy and reliability. Defense attorneys have success-
 fully argued in several  cases  that  AI-generated  evidence  should  be
 excluded  from  court  proceedings due to these issues. This act aims to
 address these concerns by prohibiting the use of AI  outputs,  including
 facial  recognition,  as  evidence while allowing their use for investi-
 gative purposes, thereby protecting the rights of defendants  and  main-
 taining the integrity of the judicial process.
   §  2.  Section  837  of  the  executive law is amended by adding a new
 subdivision 24 to read as follows:
   24. (A) PROMULGATE A STANDARDIZED AND DETAILED  WRITTEN  PROTOCOL  FOR
 THE  UTILIZATION  OF  ARTIFICIAL  INTELLIGENCE BY POLICE AGENCIES IN THE
 INVESTIGATION OF CRIMINAL  ACTIVITY.  THE  PROTOCOL  SHALL  ADDRESS  THE
 FOLLOWING TOPICS:
   (I) AUTHORIZING LAW ENFORCEMENT AGENCIES TO UTILIZE AI SYSTEMS AND FRT
 FOR  INVESTIGATIVE PURPOSES, INCLUDING BUT NOT LIMITED TO DATA ANALYSIS,
 PATTERN RECOGNITION, FACIAL RECOGNITION, AND PREDICTIVE ANALYTICS;
   (II) AUTHORIZING LAW ENFORCEMENT AGENCIES TO UTILIZE  AI  SYSTEMS  AND
 FRT  TO  ASSIST  IN IDENTIFYING POTENTIAL SUSPECTS, UNCOVERING EVIDENCE,
 AND GENERATING INVESTIGATIVE LEADS;
 
  EXPLANATION--Matter in ITALICS (underscored) is new; matter in brackets
                       [ ] is old law to be omitted.
                                                            LBD15799-01-4
              

 A. 10625                            2
 
   (III) REQUIRING LAW ENFORCEMENT AGENCIES UTILIZING AI SYSTEMS AND  FRT
 TO  IMPLEMENT  TRANSPARENCY  MEASURES,  INCLUDING MAINTAINING RECORDS OF
 AI-GENERATED OUTPUTS AND THE PARAMETERS USED  BY  THE  AI  SYSTEM.  SUCH
 RECORDS SHALL BE SUBJECT TO AUDIT AND REVIEW BY AN INDEPENDENT OVERSIGHT
 BODY  DESIGNATED  BY THE DEPARTMENT TO ENSURE COMPLIANCE WITH THE PROTO-
 COL;
   (IV) REQUIRING LAW ENFORCEMENT AGENCIES UTILIZING AI SYSTEMS  AND  FRT
 TO  UNDERGO  REGULAR  INDEPENDENT  AUDITS OF FRT SYSTEMS TO ASSESS THEIR
 ACCURACY AND BIASES. THE RESULTS  OF  THESE  AUDITS  SHALL  BE  PUBLICLY
 ACCESSIBLE AND DISCLOSED IN ANY CASE WHERE FRT EVIDENCE IS USED; AND
   (V)  REQUIRING  THAT LAW ENFORCEMENT OFFICERS UTILIZING AI SYSTEMS AND
 FRT RECEIVE THOROUGH TRAINING ON THE LIMITATIONS AND PROPER  USE  OF  AI
 SYSTEMS  AND  FRT, EMPHASIZING THE IMPORTANCE OF UNDERSTANDING AND MITI-
 GATING BIASES AND ERRORS.
   (B) FOR THE PURPOSES OF THIS SUBDIVISION:
   (I) "ARTIFICIAL INTELLIGENCE" OR "AI"  MEANS  A  MACHINE-BASED  SYSTEM
 THAT CAN, FOR A GIVEN SET OF HUMAN-DEFINED OBJECTIVES, MAKE PREDICTIONS,
 RECOMMENDATIONS, OR DECISIONS INFLUENCING REAL OR VIRTUAL ENVIRONMENTS;
   (II)  "AI-GENERATED  OUTPUT" MEANS ANY INFORMATION, ANALYSIS, OR DECI-
 SION PRODUCED BY AN AI SYSTEM DURING A CRIMINAL INVESTIGATION, INCLUDING
 BUT NOT LIMITED TO FACIAL RECOGNITION, PREDICTIVE POLICING  MODELS,  AND
 OTHER MACHINE LEARNING ALGORITHMS; AND
   (III) "FACIAL RECOGNITION TECHNOLOGY" OR "FRT" MEANS A TECHNOLOGY THAT
 ANALYZES  FACIAL  FEATURES  AND IS CAPABLE OF IDENTIFYING OR VERIFYING A
 PERSON FROM A DIGITAL IMAGE OR A VIDEO FRAME FROM A VIDEO SOURCE.
   § 3. The criminal procedure law is amended by  adding  a  new  section
 60.77 to read as follows:
 § 60.77 RULES OF EVIDENCE; USE OF ARTIFICIAL INTELLIGENCE.
   1.  ARTIFICIAL INTELLIGENCE-GENERATED OUTPUTS, INCLUDING FACIAL RECOG-
 NITION RESULTS, SHALL NOT BE ADMISSIBLE AS EVIDENCE IN ANY  CRIMINAL  OR
 CIVIL COURT PROCEEDING.
   2.  NO  PARTY IN A CRIMINAL OR CIVIL CASE SHALL REFERENCE OR INTRODUCE
 ARTIFICIAL INTELLIGENCE-GENERATED OUTPUTS, INCLUDING FACIAL  RECOGNITION
 RESULTS, DURING QUESTIONING OF WITNESSES, SUSPECTS, OR DEFENDANTS.
   3.  ARTIFICIAL INTELLIGENCE-GENERATED OUTPUTS SHALL NOT FORM THE BASIS
 FOR ANY LEGAL DECISION OR JUDGMENT.
   4. DEFENDANTS SHALL HAVE THE RIGHT TO EXPERT WITNESSES WHO CAN TESTIFY
 ABOUT THE RELIABILITY AND LIMITATIONS  OF  ARTIFICIAL  INTELLIGENCE  AND
 FACIAL RECOGNITION TECHNOLOGY SYSTEMS USED IN THE INVESTIGATION OF THEIR
 CASES.  FUNDING  SHALL BE AVAILABLE THROUGH THE OFFICE OF INDIGENT LEGAL
 SERVICES TO INDIGENT DEFENDANTS TO ACCESS SUCH EXPERTS.
   5. ALL PROSECUTORIAL OFFICES MUST DISCLOSE DETAILED INFORMATION  ABOUT
 ANY  ARTIFICIAL  INTELLIGENCE  SYSTEMS AND FACIAL RECOGNITION TECHNOLOGY
 USED IN INVESTIGATIONS, INCLUDING  ERROR  RATES,  KNOWN  BIASES,  SOURCE
 CODE, AND ALGORITHMIC TRANSPARENCY, WHERE POSSIBLE.
   §  4.  This  act shall take effect on the ninetieth day after it shall
 have become a law.