29/01/2020
Technology & Regulation in the Spotlight

The New York City Automated Decision Systems Task Force, which was established in early 2018, has released its final report on recommendations for establishing a framework for the use and policy around Automated Decision-making Systems (“ADS”).

The use of ADS (such as artificial intelligence, machine learning, data analytics, etc.) is increasing both within New York City and in other municipalities across the United States, in cities, as well as by the federal administration agencies. However, in addition to the significant benefits for the administration and citizens, the use of such technologies might introduce risks of certain implicit or explicit biases that are hard to notice and correct.

In its report, the appointed Task Force aims to provide the Mayor of New York with general recommendations that will help to better understand and manage the use of ADS and the accompanying risks. The Task Force recommends that the City of New York will establish an Organization Structure that would serve as a centralized resource for guiding agencies on the use of ADS. The Organizational Structure should develop citywide policies and best practices to assist agencies in broadly developing and implementing such automated systems. Such policies will have to provide a flexible framework that will evolve together with the advancement of technology. In addition, the task force recommends that the City will strengthen the capacity of its agencies to develop and use ADS by providing them with sufficient funding to hire experts, train personnel, and create best practices for procurement, data retention and data sharing.

Finally, the report suggests formalizing ADS management functions by establishing a framework for agency reporting and publishing of information related to ADS, providing guidelines to agencies on how to respond to public inquiries related to ADS, and to maintain an accessible public online portal through which individuals can submit questions or comment to agencies on ADS related matters.

However, the Task Force has also recognized the shortcomings of its report, being the first of its kind in largely uncharted territory. The first challenge, which the Task Force members have faced is the absence of a definition as to what is considered ADS. According to the report, the existing definition that is used is too broad and includes tools as internet search and the use of spreadsheets. In addition, the recommendations by the task force are intended for a wide range of tools or systems, including systems that do not yet exist. Consequently, the recommendations tend to be more general, having a low level of specificity. Lastly, the report is addressed solely to city agencies, whereas a significant amount of automated decision making is occurring in the private, state and federal sectors.

In this context, see our previous report on the guidelines on artificial intelligence and data protection that was published by the Council of Europe.