On 16 December the UK Government (Department for Digital, Culture, Media and Sport) published a guide to the new Online Safety Bill, including how it will protect children and adults. Much discussion has reflected on how the Bill seeks to protect children (and, in some circumstances, adults) from lawful but potentially harmful material. The guide indicates this would include, for example, online abuse and antisemitic content.
How, in practice, can people be protected given the huge amount of content posted online?
There are at least three parts to the solution which all have a role, but none of which are perfect.
People protect each other
People are integral to identifying and flagging messages, images and videos themselves. This includes users of online social networks who report content via platforms’ built-in tools. On the service’s side, teams of people may be hired to review incoming or user-flagged content to see if it breaches terms of service or relevant laws. Of course this is difficult to scale due to the resource it requires, so online services may pay other companies to employ people on low wages to moderate content. This brings its own potential harms where people are exposed to upsetting content in long and intense working days.
Machine intelligence reviews content at scale
The content users or workers flag may be used to train machine learning models. Those models may then identify patterns and classify content, such as whether it should be allowed or blocked. They are termed supervised models when the classifications are already known and the model seeks to identify content that fits. Here, then, the views of the people who flagged content may influence what is classified by the model as needing to be blocked. Alternatively, the model may use other approaches such as searching for certain words or phrases which warrant a closer look. Machine learning approaches, underpinned by human intelligence, are much easier to scale than relying on human discernment alone and reduce the exposure of human moderators to some of the most clearly harmful (and so easily machine-classifiable) content.
However, these machine learning approaches work on probabilities – they identify content that is likely to be acceptable and likely to be harmful, based on specified criteria. And these criteria certainly need to evolve over time. Just recently, I watched a video online about a horror film and the presenter used creative language to allude to the violent ways in which some characters met their ends. This will have helped the video avoid automated censoring.
But perfection is the enemy of the good
This inherent imprecision, whether humans or machines are reviewing content, means that it is perhaps impossible to protect people from all abusive content. This tempers the approach we must take, encouraging us to focus first on the lower-hanging fruit which may obviously cause emotional harm and/or lead to dangerous actions, while accepting we cannot crack the problem entirely.
The Online Safety Bill recognises that in part, since the guide indicates how adults will in some cases need to have access to tools that reduce the likelihood of them being exposed to legal but harmful content. Nonetheless, it remains to be seen how some categories of illegal content that are subtle and nuanced, such as abuse or coercion, may be effectively moderated using human and machine intelligence, given the vast amount of material posted online and the above limitations. I hope to make a small contribution to this discussion through my research, which aims to spot factors associated with conflict in online spaces.
Tim Kinnear 