We talk to Dr. Jan Kocoń from the Department of Artificial Intelligence at our Faculty about teaching AI how to recognise emotions and about the related dilemmas.

Grzelińska: Apparently, it is possible to teach artificial intelligence to recognize the emotions of a specific user.

Dr. Jan Kocoń: Yes, you can indeed do that. It is quite a difficult task. In terms of emotions and sensitivity, we are dealing with subjective phenomena. Each of us reacts individually when under the influence of emotions. We can react differently to the same stimulus.

When working on the personalization of language models, the stimuli we work with are most often texts. They can also be memes, images, recordings, etc.

– Artificial intelligence methods can be employed most effectively to solve tasks that most people solve in a similar manner. For example, if the text is in English, we expect all people to confirm it is English. As a result, the model will learn to recognize texts written in English. Similarly, most experts will recognise a given text as dealing with politics, sports or medicine – says the researcher.

Emotions are not as simple.

Exactly! If I follow the above method and want to know whether a given text, video or meme is funny or sad, the ideal situation would be for everyone to react the same way. Unfortunately, this is not the case with subjective tasks, when the situation is much more complicated. Not only because the question: “What emotions does this text evoke in you? will be answered differently by different people. People often disagree internally. One day they feel bad and see everything in black colours, so they evaluate the text as negative. On the other hand, when they are in a good mood, they may find the same text positive or even funny.

A global situation can also affect the perception of some content. We would probably evaluate the text about Russia three years ago in a completely different way than today, from the perspective of the war beyond our eastern border.

Some things shouldn't be laughed at.

Let's go back to the subjective view of reality. From our perspective, it is difficult, even challenging, to create such models that are able to understand the context of a particular person, analyse the information about him or her and select the content to suit his or her preferences. I feel that we've accomplished this task.

Today we know that language models can be trained to recognize user sensitivity. Now we ask ourselves other questions – how much does the model need to know about a person in order to be able to better determine what emotions a given content arouses in a particular person.

How many questions do you need to ask, how many surveys to complete to make the model know that?

It varies. Now we are at the stage of developing our own dataset to be used for very different subjective tasks. These are not only emotions, but also offensiveness, humour or very subjective questions that are difficult to put into these three drawers, e.g.: "Do you like this text?", "Do you understand this text?", "Is there sarcasm or irony in it?"

We already have some sets from the web, which were collected on the occasion of other projects. Interestingly, no one had previously done research on these sets and did not try to build personalized models. These are sets in which many people responded in a subjective way. The evaluated texts mainly concerned the issue of offensiveness. Scientists used these sets to build models by simply employing majority voting. If 6 out of 10 evaluators found the text offensive and 4 found it non-offensive, then a majority decision was made that the text was offensive and the model was taught that way.

Meanwhile, it turns out that only four texts are enough to create personalized models. Four random texts rated as offensive by a human are enough for the model to "feel" what content the human finds offensive.

Does that mean we are very transparent to AI?

To some extent, yes. It also depends on what phenomenon we are dealing with. In the Sentimenti project, 20,000 people rated 30,000 lexical units with "Słowosieci” in terms of overtones and emotions. In addition, they assessed 7,000 texts from the web. There were 50 ratings per text. Each of the texts was assessed in terms of 8 basic emotions from the Plutchik model, such as: joy, anger, sadness, etc. The assessed elements also included overtones: positive, negative or neutral, and the general arousal. Each text was rated on a scale. It was a very subjective task.

And what was the result?

It turned out that virtually every text was controversial. People rated them very differently. Personalization meant that for some emotions we improved the results of the task by up to 40 percentage points compared to the base model, which knew nothing about a person. So great was the discrepancy in decisions. However, it required information about a person.

In the case of offensiveness, the difference between the personalized and non-personalized models oscillated in favour of the personalized one by 5 to 10 percentage points. That's still a lot. The uncertainty that remains is our inner uncertainty

– We still don't have enough external data to improve the quality of such models. In a project named Emognition, the team of Prof. Przemysław Kazienko from the Department of Artificial Intelligence deals with recognizing emotions from physiological signals. One day we plan to combine these two worlds and at the same time use the information from our research on texts and from various types of sensors, watches, smartphones. We hope that this additional knowledge will help build a model that will be even better at knowing what we feel – comments Dr. Jan Kocoń.

That sounds great on the one hand, but a bit scary on the other.

Yes. You can have 100,000 doubts about what harm such models can do. We're trying to figure out how to use them well. Companies already use this knowledge and show ads tailored to our interests. It would be better for us to know how such solutions affect reality. Knowing what evil they can be used for, we can prevent it.

We live in a very aging society. It is possible that robots will take care of us in the future, because there will be no staff to care for the elderly. We would probably like such a robot to understand our sensitivity. So that it understands what it is that we really want, and not what most people want in a similar situation.

It is conceivable that a very lonely human would become friends with such a robot and feel less lonely. That sounds very depressing. Or maybe not?

It sounds tragic. However, on the other hand, if an alternative to a friendship with a robot is severe depression from loneliness, then a robot friend is definitely a better option. This is a step in the right direction.

Is it justified to fear that since machines understand us so well, they will take over and manipulate us in the future? Once upon a time, such fears seemed abstract. Today we consider them in terms of possible scenarios.

That's true. There are a number of dilemmas we face. We have become deeply immersed in the digital world, which helps us solve many problems, but at the same time also impairs us. If I've been using GPS all my life, I'm no longer able to use maps and road signs. On the other hand, I would like to use a GPS that will guide me on such a route that I find likeable. Or another example: I want a news website where I only read interesting press news, not nonsense.

As a result, we do not see what we are not interested in. However, we miss the chance to see something new that might turn out interesting. On the one hand, facilitation, on the other, limitation. How to avoid such restrictions? Improve awareness?

And also consciousness. However, mass personalization comes with risks. Abusive speech is already being cut out on social media. Mostly, these are texts that clearly break the law. Most of us accept it. However, this is not an ideal solution, because there is a large margin of topics which we can talk about politely, even though some people find them offensive. And vice versa – the model is already cutting something when most of us do not see anything offensive in it.

We hope that personalized models will carefully filter the web for more sensitive people in order to cut out the annoying content, but not the entire content completely. We would like it to follow the rule: "Okay, you won't see it, but let others talk about it."

The classic model currently cuts too much. Also the content that we want to discuss culturally in the forum. The risk is that we can be locked up in information bubbles. Everyone will see only their own bubble, i.e. internet utopia…

As a result, he will be convinced that the whole world is like we see it and that everyone thinks alike...

When such a person comes out of their information bubble and comes into contact with reality, they will experience a shock. They will not be able to react normally. I have already seen several scientific papers devoted to this. I hope that our research on personalization will enable us to find a way to allow information from outside the bubble, so that people can painlessly open up to the perspective from the outside and avoid shock.

As a society, we must learn to live among people who have different, sometimes very different, views. We hope that research on personalization will help us find a perfect balance between human preferences and surprising content from outside the bubble. And that this balance will help us as a society to come closer together. It will make us understand each other better.

Interview by: Agata Grzelińska