Regulating high-risk artificial intelligence (AI) systems is an urgent issue, yet technical infrastructure for their effective regulation remains scarce. In this paper, we address this gap
by identifying key challenges in developing technical frameworks for AI systems’ regulation and proposing conceptual, methodological, and practical solutions to address these challenges. In this regard, we introduce the concept of AI’s operational qualification and propose the temporal self-replacement test, akin to certification tests for human operators, to examine the AI’s operational qualification. We propose measuring AI’s operational qualification across its operational properties critical for its regulatory fitness and introduce the operational qualification score as a pragmatic measure of AI’s regulatory fitness. In addition, we design and develop a Secure Framework for AI Regulation (SFAIR), a tool for automatic, recurrent, and secure examination of an AI’s operational qualification and attestation of its regulatory fitness, leveraging the proposed test and measure. We validate the efficacy of the temporal self-replacement test and the practical utility of SFAIR by demonstrating its capability to support regulatory authorities in automated, recurrent, and secure AI qualification examination and attestation of its regulatory fitness using an open-source, high-risk AI system. Finally, we make the source code of SFAIR publicly available.
Haroon is a Lecturer at Chalmers University. Previously, Haroon was a visiting research scholar at the Department of Computer Science and Engineering, Southern University of Technology, China, where he worked on AI regulation. He identified challenges to effective regulation of AI systems and proposed conceptual, methodological, and practical solutions to address these challenges. Before that, he had a postdoctoral position at Umeå University, Sweden. He conducted research in software security with a particular focus on fuzzer evaluation. He discovered serious flaws in state-of-the-art fuzzer evaluation benchmarks and proposed mitigations. He completed his doctoral degree at the School of Computer Science, Guangzhou University, China. The focus of his PhD research was on data over-collection in Android smartphones.
His research interests include identifying and solving privacy, security, and trust issues of emerging technologies in the rapidly changing privacy and security threat landscapes. He is also interested in identifying and solving problems arising from placing the solutions based on these technologies in operational settings. He has published over twenty articles and conference papers at reputed venues, including IEEE TDSC, IEEE TCSS, IEEE IoT Journal, IEEE TVT, Information Sciences, Computer Standards and Interfaces, and Neurocomputing.