Blog 15: Named Entity Recognition and Historical Texts

By Felix Vanden Borre

VOICES is experimenting with different computational methodologies to query historical sources at scale and recover the lives of non-elite early modern Irish women. One such methodologies is Natural Language Processing (NLP). This is the branch of artificial intelligence that focuses on natural (i.e. human) language. Computers are great at scrunching numbers, but less effective at processing texts. This is where NLP models come in. Algorithms can be taught to recognise linguistic features such as syntax and semantics.  Some of these models are trained specifically to recognise named entities in text. 

Named Entity Recognition (NER) is a sub-technique of NLP, which can prove particularly useful to historical research. Named entities can be people, places, national, political or religious groups, currencies, dates, work of art, and so forth. NER can thus be used to perform distant reading, rapidly identifying points of interest and patterns across large datasets. However, what is gained in efficiency with NER inevitably comes at the cost of some accuracy. Even the best models miss out on certain entities. An NER model is usually considered state of the art if it reaches 90% accuracy or more.  

This is why VOICES makes use of human-in-the-loop approaches and controlled vocabularies when using NER to ensure that no relevant information is glossed over (read more on Blog 5: Giving Voice to the Past: How AI Helps Us Listen). These processes also help us deal with the other layers of complication that VOICES faces in applying computational methods to historical data.  

One major challenge is that off-the-shelf NER models are typically trained on modern datasets. When deployed on early modern texts that contain different orthographical and typographical conventions, such as the Funeral Entries, these models can struggle with unfamiliar data. If Catherine is spelled “Katerin” and Margaret is abbreviated to “Margt.”, these women might be missed as entities. The same applies to specific textual domains. Off-the-shelf models may, for example, not have been trained on large amounts of Irish names. This can contribute to further silencing the women that VOICES is interested in. 

See the following extract from GO MS 66 about the alleged murder of Mary Babington, dowager Lady Dunsany, by her servant Honora ny Caffery (Honora Ní Caffery).  This funeral entry records the crime and the response to it. Honora was burnt for this petty treason. However, her name was cleared post-mortem when a “knave” (an early modern word for thug) confessed to the murder.  

As you can see from the NER, Dunsany, a barony, is not correctly recognised as a place. Honora ny Caffery, the alleged murdered of the Lady dowager, is also not correctly grabbed as one person by the off-the-shelf models, presumably because her surname is not an English-language one. 

The very process of generating our data using Handwritten Text Recognition also carries some risk of mistranscription (read more on Blog 7: From Handwritten Funeral Entries to Machine-Readable Texts – An internship opportunity using Transkribus to perform HTR). This is not a major obstacle; preliminary analysis suggests that many entities are correctly recognised despite being mis-transcribed. However, recognising an actual entity versus a mistranscription can be tricky. In the below image, the words “de Votre” were identified from ink that has seeped through from the other side of the paper. After NER processing, “Margerie de Votre” was marked as an entity, when in truth only “Margerie” appears in the funeral entry.  

Human-in-the-loop verification is therefore a fundamental part of the machine-learning processes deployed in VOICES. On top of controlling the accuracy of computational methods in early modern contexts, understanding how off-the-shelf models behave is crucial in balancing the efficiency offered by distant reading and the accuracy required by historical research.  

Many of the sources used by VOICES are incredibly rich yet underexplored. They are also very complex. The Funeral Entries record several thousand deaths, many of women whom we know very little about. Finding ways to deploy Named Entity Recognition on this corpus as accurately as possible using a ‘human in the loop’ is an opportunity to recover these voices and facilitate their restitution to the historical record.