Automatic Detection of Characters in Case Insensitive Text in Comics

Huge amounts of comics are being published on a daily basis. However, the textual data that are circumscribed in comics’ speech balloons and thought bubbles are highly unstructured, and very hard to automatically extract and digitize. Several automatic techniques have been used to automatically detect the shapes, sizes, and contents of speech balloons and thought bubbles leading to automatic text digitization and localisation in comics (Rigaud et al 2013; Ho et al 2012). Such advances open new horizons for a myriad of text mining applications to comics, namely, automatic indexing, searching, recommendation and visualization.

Named Entity Recognition (NER) is a task of information extraction under text mining that aims to identify in-text references to concepts such as people, locations and organizations, mainly in unstructured natural-language text. NER is very useful for text indexing, text summarization, question answering and several other tasks that enhance the experience between humans and literature.

In a previous study, we developed a simple and original method of Unsupervised Named Entity Recognition and Disambiguation (UNERD) (Mosallem et al 2014) to automatically extract names of people, locations and organizations from French and English (Abi Haidar et al., 2016) newspapers. We then used the text coordinates in the ALTO XML format to automatically locate and highlight the named entities detected by UNERD on the scanned image of the newspaper (Abi Haidar et al 2014). Last, we used UNERD to detect and visualize named entities from French newspaper during the period of the first world war (Abi Haidar et al 2016).

The main challenges encountered in unsupervised NER have been identified and addressed with our original UNERD method when applied on English and French newspapers and French literature. These challenges include but are not limited to named entity disambiguation, named entity boundary detection, and domain-specific dictionary attribution. In comics, in addition to the aforementioned challenges, the case-insensitive text extracted from comics adds the challenge of proper noun detection that we used to handle using POS tagging. We used TreeTagger (Shmid 2005) POS tagger to detect proper nouns, however and like other POS taggers, it works only when applied on case sensitive text.

Here, we use a variation of our UNERD method to detect names of fictional characters in a  database of unstructured text from comics that has been recently digitized by our partners at the L3i Labs using an active contour model for speech balloon detection (Rigaud et al 2013). Au lieu of POS tagging for proper noun extraction, we filter stop words that constitute a 5000 word list of most frequent French words. For the dictionary, we use  the freebase dictionaries of fictional characters and characters in fiction novels from Freebase. The comics digitised data amounts to around 8000 case-insensitive words in the French language. We evaluate our method's precision by analysing the predicted character names.

Out of 182 predictions of character names made by UNERD, 113 names were correctly predicted. This precision of 62% cannot be compared to that obtained in our previous results with UNERD when tested on case sensitive textual data. We also did not compute the coverage since we do not have any gold standard or any annotated data. In the table below, we see the most frequent characters that were automatically detected. The terms ‘FLIP’ and ‘HUM’ could be discarded by adding a list of onomatopoeic terms to the list of stop words thus improving precision to 68%. Our precision compares well to results presented in Cornolti's meta-study of Named Entity Recognition/Entity Linking resulting in a precision of 69% (Cornolti et al 2013).The extracted character names are then associated with page numbers to help with the automatic indexing of characters that is otherwise very costly in time and human resources. Co-occurrences of character names in the same page are used to detect dialogues or interactions between the characters. Such information, along with the mere occurrences of character names, are used by our partner, Actialuna, in order to enhance digital comics recommendation. Our method can be applied to extract entities from other case-insensitive texts by simply adapting the dictionary to the targeted text. Our results are preliminary and have been tested on a tiny database that is currently being updated.