Design and governance recommendations

This work package formulates design and governance recommendations aimed at safeguarding and strengthening human agency in socio-technical ensembles involving AI. The focus is on identifying the technical and organisational features that allow users to remain meaningful participants in human–AI collaborations, rather than passive recipients of algorithmic outcomes.

Leads: Célia Zolynski (Université Paris 1 Panthéon-Sorbonne), Raja Chatila (ISIR) Contributors: Carina Prunkl, Mehdi Khamassi, Anouk Barberousse, Mohamed Chetouani

Two subtasks

1. Recommendations to enhance human agency — starting from the “effective human oversight” required by Article 14 of the EU AI Act, specifying what such oversight entails in practice: situational awareness, traceability, reversibility, contestability; and linking these criteria to interface design, feedback mechanisms and accountability structures.
2. Improving the law to protect human agency — beyond the AI Act and the DSA, exploring economic regulation mechanisms that take the human relational context into account; examining whether an impact on human relationships can count as a systemic risk within the meaning of the AI Act; studying how new rights proposed for digital interfaces (a right to parameterisation, a right to interoperability) could apply to conversational agents.

The expected outcome is a white paper of design and governance recommendations, illustrated with case studies, to inform both the development of next-generation systems and the evolution of regulatory frameworks.