Tim Kinnear

Social media brings people with opposing views into contact more frequently than in other aspects of our social lives. These interactions can generate more heat than light. Why might this happen? This post explores some reasons.

Discussion on this subject often focuses on how social media platforms’ algorithms recommend content to encourage engagement with their platform. People may be more likely to interact with something that sparks a fire within them, so platforms connect users along these lines. However, those who write these algorithms may not intend negative interactions. Content recommendation can still connect people with polarised views even if it’s not intentionally set up that way. This is down to how people’s behaviour is used to recommend content to them. Association rules mining (ARM) is a machine learning technique often used here. ARM looks for ‘item sets’ which identify that every time a person using the service carries out a certain action, there is a range of things they could do next, each of which have different probabilities. For example, on a film streaming website perhaps two horror films – the second having themes similar to the first – form a common item set as people often watch the second film after they watch the first. Therefore, the platform will recommend the second film as the first one finishes, increasing use of the service and satisfying subscribers. These item sets are based on datasets of users’ historical interactions. On social media, if a person engages with certain accounts or messages and then gets riled by a second account with opposing views and interacts with it, other followers of the first account may be served content from the second. Here, a seemingly innocuous means of recommending content has increased exposure to opposing, perhaps antagonising messages.

The underlying structure of the internet also connects people in ways that other ‘offline’ social networks may not. Away from social media, people form groups of friends and other social networks by choice, resonating with those similar to them. Birds of a feather flock together. Of course the internet connects people who are dispersed spatially around the world, cutting across these chosen networks. Further, the underlying technologies – such as packet switching which routes data across decentralised online networks – afford near real time communications between large networks of people in ways that predecessor media technologies such as phone and radio simply did not. Connections are less linear than before and less within participants’ control.

In addition, people may connect online based on a topic of discussion, rather than their views on it. In previous generations, joining clubs or associations often meant aligning with a certain political party, sports team or cause. Now people connect via hashtags, social media groups and populate online discussion boards based around certain topics, notwithstanding the diversity of views they hold around them. This brings people into contact who have little in common save for the discussion topic, and the full richness of other participants’ lives are lost in the online environment which foregrounds the topic at hand and how people differ on it.

So how might we respond to these challenges to ensure our online interactions remain engaging and useful? In a future post, I’ll review approaches that moderators, platform owners and social media participants have taken in the past to encourage healthy interactions online while also holding in tension the need for people to express themselves authentically.

Machine learning has attracted much attention in recent years as researchers and developers have extended its capabilities to classify and predict human language. This powers diverse technologies such as smartphone digital assistants, internet search engines and social media content moderation.

Indeed, diverse uses for AI have been of longstanding interest to researchers, as reflected in discussions of AI chess among academics and scientists on early online social networks. However, what has changed is the scale of computing power which enables AI to be taught using very large datasets and undertake more complex computation. In the case of chess, this led to AI trained on large numbers of historical chess matches between people, then later through high numbers of training games in which the AI plays itself and develops strategy iteratively. The book Game Changer (2019) provides a detailed and fascinating account of recent developments.

In the case of AI which dialogues with users in human language, training often involves analysing text online to learn how language is used. This results in an AI model, which is to say an abstract representation of how the components of the language relate to one another. The model is often then refined through automated iterative testing. This can include masking some proportion of words in a given sentence and having the model predict the masked words or considering which of multiple sentences follows a certain sentence from the training data. The test results are then used to adjust the model and refine how it captures relationships between different words.

Online text can shape how AI responds to people

A concern arises when AI exhibits inappropriate biases in its responses. One reason for this is how it is trained. The language fed into the model during training informs how it predicts a suitable response when it is later used in services such as digital assistants and internet search engines. Crucially, and consequently, machine learning models may be shown to reproduce biases in society, such as replying stereotypically when asked to predict a man’s career and a woman’s career.

Here, text has power in two ways. First, a vulnerable person engaging with these AI driven services may receive responses that reinforce certain unhelpful perceptions regarding their status, identity or the possibilities for their lives. This could impact a person’s choice, self-esteem and wellbeing more generally. Second, this means the language we use online can, taken together, affect how others experience bias or inclusion in their interactions with AI services.  This illuminates how AI has the power to amplify both negative and positive ways in which people are affected by what others say. By using thoughtful and inclusive language, we can help steer that power toward correcting and avoiding historical discrimination.

On 16 December the UK Government (Department for Digital, Culture, Media and Sport) published a guide to the new Online Safety Bill, including how it will protect children and adults.  Much discussion has reflected on how the Bill seeks to protect children (and, in some circumstances, adults) from lawful but potentially harmful material. The guide indicates this would include, for example, online abuse and antisemitic content.

How, in practice, can people be protected given the huge amount of content posted online?

There are at least three parts to the solution which all have a role, but none of which are perfect. 

People protect each other

People are integral to identifying and flagging messages, images and videos themselves. This includes users of online social networks who report content via platforms’ built-in tools. On the service’s side, teams of people may be hired to review incoming or user-flagged content to see if it breaches terms of service or relevant laws. Of course this is difficult to scale due to the resource it requires, so online services may pay other companies to employ people on low wages to moderate content. This brings its own potential harms where people are exposed to upsetting content in long and intense working days.

Machine intelligence reviews content at scale

The content users or workers flag may be used to train machine learning models. Those models may then identify patterns and classify content, such as whether it should be allowed or blocked. They are termed supervised models when the classifications are already known and the model seeks to identify content that fits. Here, then, the views of the people who flagged content may influence what is classified by the model as needing to be blocked. Alternatively, the model may use other approaches such as searching for certain words or phrases which warrant a closer look. Machine learning approaches, underpinned by human intelligence, are much easier to scale than relying on human discernment alone and reduce the exposure of human moderators to some of the most clearly harmful (and so easily machine-classifiable) content.

However, these machine learning approaches work on probabilities – they identify content that is likely to be acceptable and likely to be harmful, based on specified criteria. And these criteria certainly need to evolve over time. Just recently, I watched a video online about a horror film and the presenter used creative language to allude to the violent ways in which some characters met their ends. This will have helped the video avoid automated censoring.

But perfection is the enemy of the good

This inherent imprecision, whether humans or machines are reviewing content, means that it is perhaps impossible to protect people from all abusive content. This tempers the approach we must take, encouraging us to focus first on the lower-hanging fruit which may obviously cause emotional harm and/or lead to dangerous actions, while accepting we cannot crack the problem entirely.

The Online Safety Bill recognises that in part, since the guide indicates how adults will in some cases need to have access to tools that reduce the likelihood of them being exposed to legal but harmful content. Nonetheless, it remains to be seen how some categories of illegal content that are subtle and nuanced, such as abuse or coercion, may be effectively moderated using human and machine intelligence, given the vast amount of material posted online and the above limitations. I hope to make a small contribution to this discussion through my research, which aims to spot factors associated with conflict in online spaces.