Second Look: gender and sentiment on show

What are the most frequent sentiments identified by AI for the given selection of portraits?

When AI assesses a subject’s sentiment, it provides confidence ratings for the following emotions: happy, sad, calm, angry, disgusted, confused, surprised, and fear. Interested in revealing at a high level the sentiment of the portrait collection, the below visualization groups the paintings according to their most dominant emotion or emotion with the greatest confidence rating. The size of each circle is relative to that confidence rating, so a “SAD” painting with a 92% confidence score for that emotion will be larger than a “SAD” painting with a similar score of 44%. As can be seen, this visualization reveals that the portrait collection is predominately “CALM” according to AI.

Each circle represents a painting.
Circle sizes are relative to AI's confidence that the painting portrays the given sentiment.

How confident is AI in its sentiment analysis of these paintings?

AI only "sees" (separately) binary genders

When AI assesses a subject’s gender, it provides a confidence rating for how sure it is about which gender the subject is. Its analysis is purely based on a subject’s physical characteristics, as AI cannot infer a person’s subjective gender identity. Therefore, a higher confidence rating may be correlated with AI recognizing greater outward masculinity or femininity in a subject. This speculative correlation is visualized here via the circle rays.

The double ring circumscribed by the white circle visualize the different relationships between the gender categories of female, male, both, and neither, which are umbrella terms for an array of gender identities. The rays extending beyond the white circle show the frequency that AI recognizes a subject’s gender for a given confidence rating between 1% and 100%. Reflecting the logic of the double ring, a 1% confidence correlates with “less masculine” or “less feminine” and a 100% confidence correlates with “more masculine” or “more feminine” hence the layout of the rays. Because the AI used here has not been trained to "see" non-binary genders, there are no rays extending from the top or bottom sections of the chart.

What are the most prominent colors for specific genders and sentiments?

Interested in revealing at a high level the colors associated with a given gender and/or emotion in the portrait collection, this visualization first observes each painting’s most dominant color, then creates a frequency chart in the form of a color bar. By clicking on one of the colors within the bar, the mosaic chart below the bar is updated with images of the paintings whose dominant color matches the selected color. By selecting “HUE” from the lefthand filter menu, all of the above can be replicated using the dominant hues of the paintings.