Contents
Goals
- Exploring social and political impact of different forms of AI application.
- Applying risk assessment criteria
- Familiarise with the variety of application of AI in the society
Steps
Preparation
If you use the template: Print it out and cut out the different applications. Each group would receive one set.
1. Introduction
Because cars are inherently more dangerous than bicycles, there are more regulations surrounding their production, sale, or operation than there are for bicycles. The more risky and complex a system is, the more closely we must examine it and learn to understand it. In this sense, we can cluster AI systems on a line between low-complexity and high risk. The task is to identify various applications of AI in business, government, society and private life and to sort them along these lines.
2. Collection
Option A: Collect in a plenary the applications of AI in state, society, economy and houseolds that you know. Write each on one card/post it.
Option B: Share the examples from the template and ask the particiants to mention further examples of AI applications and systems they know.
Encourage participants to think not only about ChatGPT or other current AI solutions, but also about those that are not so widely discussed.
3. Clusters
Or in a plenary setting or in smaller groups, the participants should group the applications according to their level of risk and explain how they would assess them.
- What risk are you referring to? Why?
Risks affecting
- personal life and autonomy
- health
- the rule of law
- rights of individuals and groups
- financial damage
- security
- creating conditions that facilitate abuse of power or corruption
- ...
4. Risk Mitigation
After clustering, (maybe in smaller groups focusing on some cases): Come up with ideas for risk mitigation. What would have to happen for the risk/threat to be lower?
- Mitigation of risks through guidelines and rules: Risk assessment, security measures, transparency of collected data for users, deletion policy...
- Limiting technical features, strengthening user rights, more sharp laws and regulations, or certain use cases...
- If helpful - distinguish between systemic risk mitigation and on other levels (for groups, or individuals...)
Example: Biometric access to one’s personal smartphone is OK, but gaining access with the smartphone owner's iris photo is highly questionable. Also tracking a person based on biometric features and to analyse their movements and actions in real time by systems is problematic regards different of the before mentioned risk categories. If this were implemented on a widespread basis, there would be a risk of total surveillance on systemic level.
5. Transfer: A risk-based approach
Explain that lawmakers in the EU also considered the risks and problems associated with AI systems. Designing the AI Act the followed a risk-based approach. For example, providers of high-impact AI applications are required to conduct more thorough risk assessments and reporting. Some applications are prohibited, while others are only permitted under certain conditions.
The AI Act defines 4 levels of risk for AI systems: Unacceptable risk; High-risk; limited risk, minimal or no risk
Contrary to what this pyramid might suggest, classifying an AI application into one of the lower categories does not mean that it is less problematic for individuals. On the contrary: we are increasingly confronted with AI applications whose impacts are not assessed or recognized as systemic, but which can have a profound effect on many of us as individuals.
Unacceptable Risk
Deploying subliminal techniques beyond a person’s consciousness or purposefully manipulative or deceptive techniques, manipulating decision-making and behaviour
Exploiting vulnerabilities of a person or a specific group due to their age, disability or a specific social or economic situation, distorting the behaviour and causing harm
Evaluating or classifying persons or groups of persons based on their social behaviour or personality characteristics and discriminating them (social scoring)
Assess or predict the risk of a person committing a criminal offence, based solely on the profiling of a person or on assessing their personality traits and characteristics (allowed: support of human assessment with data directly linked to a criminal activity)
Creating or expanding facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage
Inferring emotions of a person at work or in education institutions (allowed: medical or safety reasons)
Biometric categorisation regards sensitive data to deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation (allowed: ‚lawfully acquired biometric datasets‘ in the area of law enforcement)
Remote biometric identification in publicly accessible spaces for law enforcement (eventually allowed: targeted search for specific victims, prevention of a threat, localisation or identification of certain suspected criminals)
Real-time remote biometric identification for other purposes and more than confirming the identity of a specifically targeted individual.
- Source: AI Act Article 5: Prohibited AI practices
High-risk AI applications defined in the AI Act
Biometry
Remote biometric identification systems (not those for the sole purpose of which is to confirm that a specific natural person is the person he or she claims to be);
Biometric categorisation systems, according to sensitive or protected attributes or characteristics based on the inference of those attributes or characteristics;
Education
Emotion recognition; access or admission or assignment; evaluation of learning outcomes, including when those outcomes are used to steer the learning process of natural persons; assessing the level of education; monitoring and detecting prohibited behaviour of students during tests.
Work & employment
Recruitment or selection of natural persons (targeted advertisement, filtering job applications, evaluate candidates); decisions affecting terms of work-related relationships, employment contracts, to allocate tasks based on individual behaviour or personal traits or characteristics, monitor and evaluate the performance and behaviour of employees.
Migration
Assess a risk, including a security risk, a risk of irregular migration, or a health risk, posed by people during migration into EU; Examination of applications for asylum, visa or residence permits; Detecting, recognising or identifying natural persons (allowed: verification of travel documents).
Law enforcement
Assess the risk of a natural person becoming the victim of criminal offences; usage of polygraphs or similar tools; evaluate the reliability of evidence in the course of the investigation or prosecution; assessing the risk of a natural person offending or re-offending, assessing personality traits and characteristics or past criminal behaviour of natural persons or groups; profiling of natural persons;
Justice
Researching and interpreting facts and the law and in applying the law to a concrete set of facts, or to be used in a similar way in alternative dispute resolution; influencing the outcome of an election or referendum or the voting behaviour of natural persons in the exercise of their vote in elections or referenda (allowed: AI systems such as tools used to organise, optimise or structure political campaigns from an administrative or logistical point of view).
- Source: AI Act Annex III to Article 6 (2): High-risk AI applications
Reflection
- How easy was the task? Where did you have to think longer or did you not reach a conclusion?
- How do different groups in the population view the respective application? What rights should those affected have?
- Is it better to be very risk-aware and take preventive measures, or to react quickly when something happens?
- What do the application possibilities trigger in you? Have you already gained experience with them?
- What can you contribute to minimising risk, who else is involved? Who should be involved in governing/controlling these AI applications?
- Whose perspective on risk definitions is considered in Europe? Are fundamental rights at core of risk assessment?
Handbook: More than Go with the Flow
- A handbook on Digital Citizenship Education, created in the frame of the project DIYW-ROAD/Competendo. Digital Youth Work - rights-sensitive, open, accessible, democratic.
- Unless otherwise stated, authors and editors of the methods published in the project are Elisa Rapetti, Markus José Plasencia Kanzler, and Nils-Eyk Zimmermann
