The European AI Act: Challenges and Opportunities for Developers and Users of Affective Systems
On 13 March 2024, the European Parliament approved the European AI Act. This act is an important instrument to promote the responsible use of AI, to prevent abuse and to create a safe society for all Europeans. I fully concur with the importance of citizens’ privacy, systems’ transparency and the need to monitor AI systems’ operation to signal and act upon possible unexpected and unwanted effects of these systems. However, the new regulation contains a clause that is potentially very damaging for our field, i.e. the categorical prohibition of emotion recognition in the workplace and in education institutions. Fortunately, an exception is made for AI systems developed for medical or safety reasons. This is good news for developers of drowsiness detection systems in the aviation and automotive industries. However, it leaves many applications of emotion recognition that enhance comfort, wellbeing and health in the workplace and educational institutions in the prohibited category. Examples of such applications include: affective tools that improve the usability and accessibility of digital systems, systems that adapt to the mental state of the user in order to achieve a balanced workload and prevent burnout, affective systems that enhance the quality of interaction during video conferencing, and systems that help call center operators to respond adequately to customers’ emotions. In January and February 2024, over 120 research institutes and individual experts across the EU co-signed a letter to key persons in the European Commission, Parliament and Council, urging them to reclassify emotion recognition from prohibited to high-risk. This did not happen, but awareness for our case has been created. The dialogue continues with the newly established European AI Office, which will develop the Guidelines for Implementation of the AI Act. Finally, these guidelines must be translated by each member state into local regulations. Only then will the implications of the AI Act become fully clear for developers and users of affective systems. In the meantime, our research community should remain alert and involved, helping European and national policymakers with fine-tuning the guidelines with respect to emotion recognition, so that Europe can reap the benefits of these promising technologies while risks are mitigated.
Lucas P.J.J. Noldus
Prof. Dr. Lucas P.J.J. Noldus is founder and CEO of Noldus Information Technology, a developer of software tools and integrated measurement systems for the study of human behavior and human-system interaction, headquartered in The Netherlands. Noldus’ systems have found their way into thousands of academic and industrial research labs around the world. Lucas holds an MSc in biology from Leiden University (1983) and a PhD in behavioral ecology from Wageningen University (1989). In 2019 he was appointed professor in Behavior, Information Technology and Innovation at Radboud University, Nijmegen. He has (co)authored more than 140 papers and conference presentations about methods and techniques in behavioral research. In 2023 he was elected as Fellow of the Netherlands Academy of Engineering. Besides his corporate and academic work, Lucas serves on a range of boards and committees related to science, innovation and sustainability.