Collecting Judgments on Artworks Through a Similarity Game

We present PAGANS (Playful Art: a GAme oN Similarity) ¹ a playful activity to be performed by pairs of users in order to collect similarity judgments about artworks. The final goal of this task is to have indicators concerning how people perceive artworks and how they judge their similarity. We are also interested in comparing such judgments with the opinion of art curators, and see whether users’ contribution can be integrated in the arrangement of a virtual or physical exhibition in view of a crowd-curation approach (Ridge, 2014).

PAGANS foresees the involvement of a pair of users at a time, who play in parallel. Each of them plays the same game independently: similarity was not explained and participants were asked to follow their intuition. A final score, presented as a sort of “aesthetic affinity score”, is obtained by comparing the two judgments and how much they overlap. The game could be played online, but so far the collection method has been tested in real-world scenarios, where both players are physically in the same place and one researcher is available to give feedback after the completion of the activity.

The game itself is as follows: a virtual card representing an artwork is given (the card with a red pin on the right of the table in Fig. 1), while a set of other 10 cards is displayed to the user. (S)he has to drag and drop on the round target the cards in order of similarity to the given card, until all images on the table are ranked.

Fig. 1: Game interface

When both players have completed the task, they enter information about gender and age, and then the system shows the dashboard displayed in Fig. 2. The Pearson’s, Spearman’s and Kendall’s coefficients (Hauke and Kossowski, 2011) are three metrics that measure with slight differences the players’ agreement on the similarity judgments (the higher the value, the higher the agreement). This score is presented to the players as their aesthetic affinity score. The best affinity is reached when the players choose the same ranking, since there is no ‘gold standard’ order. Players’ affinity is also compared with the ones displayed on the right of the dashboard: the average score obtained by other pairs previously participating in the game (“Overall correlation”), the average agreement among all male players, and that of female players. Another useful information is the “Rank switching trend”: for each artwork to be ranked, the picture shows if the two players put them in the same order (straight line) or if they switched some positions.

Each picture displayed in the game was pre-processed with the LIRE tool ² that extracts automatically image-related features such as color and shape. These features are used to provide information about the similarity judgments provided by the players, specifically if similarity relies more on color or shape information. Finally, the system outputs on the fly a network, where each node is one of the artworks in the game and the distance from the central node (i.e. the pinned card) is proportional to the average rank assigned by the two players.

Fig. 2: Players dashboard

PAGANS was presented during Researchers’ Night 2015 in Trento (Italy) as a game for pairs of players, allowing us to collect around 170 game sessions in few hours. The game environment proved successful in engaging players also thanks to some gamification strategies. For instance, every hour the system automatically displayed a message assigning two free museum entrances to the players currently involved in the game. Besides, we kept track of the best affinity scores, and we identified the “winning pair” of the night. This boosted competition, with players trying to beat the highest score.

The goal reached with this first experiment was two-fold: on the one hand, some verbo-visual works from “Archivio di Nuova Scrittura” (Ferrari, 2012) were displayed in digital form for the first time, reaching an audience that would not necessarily see them in an exhibition. Since the game included four possible similarity sessions, around 50 artworks were shown. These works are usually kept in the archive of MART ³ and MUSEION ⁴ museums, not visible to the public. A second advantage of PAGANS is that we were able to collect in a short time several similarity judgments, which will be used to investigate which features related to images and possibly to persons’ age and gender correlate best with similarity. These analyses are currently in progress.