A Web-based Tool Called Gauntlet: From Iterative Design To Interactive Drawings Annotation

Gauntlet was developed for the international SNSF project “Drawings of gods”, which is dealing with up to 5’100 drawings collected in different parts of the world (e.g. Japan, Russia, Switzerland, Romania). The data was digitalized and deposited on a publicly available database designed on MySQL for this project ( http://ddd.unil.ch). The presentation of this tool will then make use of a specific application on such material.

Subsequent analyses involve interpretation of collected data and metadata (Brandt et al., 2009; Dandarova, 2013) as well as image analysis (Konyushkova et al., submitted). While algorithms and software tools for automated image analysis of photographic pictures are steadily expanding, high-variability and approximate nature of children’s drawings make analysis and object detection (e.g. stars, clouds, eyes) rather challenging.

Convinced that the curation (i.e. manual annotation) of our complex objects is the solution both to tackling such a big data problematic and to guiding further automated computational approaches, we built Gauntlet, a web application tool for image annotation. It results from a collaborative and interdisciplinary software development process gathering psychologists, theologians and bioinformatics specialists. The tool is available for Chrome (version 44.0 or higher) and Firefox (version 41.0.2 or higher). It was built on modern and open source web standards for software development such as HTML5/CSS3 and AngularJS. There is no restriction to the use of this tool and it is GNU GPLv>=2. The logic behind follows from web tool development in bioinformatics, aiming to provide platform-independent and barrier-free solutions - which could not be found with existing tools.

Gauntlet's main benefit is to provide our worldwide collaborators with common semantics from a set of attributes (i.e. features) and geometrical tools for annotating the collected drawings. The set is displayed on the interface as a hierarchical list - termed “annotation tree” - of categories and subcategories (e.g. “characters area" branch contains sub-branch attributes: “human”, “non-human”, “text” and “blank sheet”). The tree is designed and edited via Excel and converted to JSON for web display. Such an implementation grants us flexibility and editing of the annotation tree based on curators’ comments (full text comments can be dropped at every level) and the degree of attributes occurrences. The annotation tree first took after McCarty et al.’s personification model (2004) approach, but is currently fashioned in such a way that the role played by annotators’ subjectivity should be reduced.

While it allows relatively fast annotations of large samples of drawings, Gauntlet’s main purpose is to provide positional coordinates of attributes from smooth and accurate user-image interactions. Such interactions rely on points to target repeated objects of a specific attribute (e.g. accessories or hands) and boxes to mark off the whole spanning area of an attribute (e.g. characters area). Curators can, at any time, export the current status of annotation in CSV format and run comparative statistical analysis on feature frequency, occupancy or positioning. So far, annotations have been carried out by researchers who are affiliated with our project, although crowdsourcing may be used in the future.

In the end, it is expected to help highlight developmental, intercultural and interfaith variations on the material collected for the abovementioned project “Drawings of gods”. Moreover, exports from Gauntlet will help complete further explorations into pattern recognition by providing the ground truth.

At this stage the tool is client-based and main features are ready for use. Even though it is stand-alone, it may go through further development to plug into API for operating with any kind of data storage systems (e.g. RDBMS, nosql or rdf-based). Beside this, due to its flexibility such a tool might prove practical for a wider use, targeting the processing of data in various contexts, such as photos of paintings or other art forms.

Overall, the collaborative and interdisciplinary work from which Gauntlet emerged represents a great deal of mutually enriching influences between agents from various fields ready to revise their initial views, thus shedding light on what today's scientific project development should look like.