Machine learning has attracted much attention in recent years as researchers and developers have extended its capabilities to classify and predict human language. This powers diverse technologies such as smartphone digital assistants, internet search engines and social media content moderation.
Indeed, diverse uses for AI have been of longstanding interest to researchers, as reflected in discussions of AI chess among academics and scientists on early online social networks. However, what has changed is the scale of computing power which enables AI to be taught using very large datasets and undertake more complex computation. In the case of chess, this led to AI trained on large numbers of historical chess matches between people, then later through high numbers of training games in which the AI plays itself and develops strategy iteratively. The book Game Changer (2019) provides a detailed and fascinating account of recent developments.
In the case of AI which dialogues with users in human language, training often involves analysing text online to learn how language is used. This results in an AI model, which is to say an abstract representation of how the components of the language relate to one another. The model is often then refined through automated iterative testing. This can include masking some proportion of words in a given sentence and having the model predict the masked words or considering which of multiple sentences follows a certain sentence from the training data. The test results are then used to adjust the model and refine how it captures relationships between different words.
Online text can shape how AI responds to people
A concern arises when AI exhibits inappropriate biases in its responses. One reason for this is how it is trained. The language fed into the model during training informs how it predicts a suitable response when it is later used in services such as digital assistants and internet search engines. Crucially, and consequently, machine learning models may be shown to reproduce biases in society, such as replying stereotypically when asked to predict a man’s career and a woman’s career.
Here, text has power in two ways. First, a vulnerable person engaging with these AI driven services may receive responses that reinforce certain unhelpful perceptions regarding their status, identity or the possibilities for their lives. This could impact a person’s choice, self-esteem and wellbeing more generally. Second, this means the language we use online can, taken together, affect how others experience bias or inclusion in their interactions with AI services. This illuminates how AI has the power to amplify both negative and positive ways in which people are affected by what others say. By using thoughtful and inclusive language, we can help steer that power toward correcting and avoiding historical discrimination.
Tim Kinnear 