Jul 31st 2015

Which paintings were the most creative of their time? An algorithm may hold the answers

From Picasso’s The Young Ladies of Avignon to Munch’s The Scream, what was it about these paintings that arrested people’s attention upon viewing them, that cemented them in the canon of art history as iconic works?

In many cases, it’s because the artist incorporated a technique, form or style that had never been used before. They exhibited a creative and innovative flair that would go on to be mimicked by artists for years to come.

Throughout human history, experts have often highlighted these artistic innovations, using them to judge a painting’s relative worth. But can a painting’s level of creativity be quantified by Artificial Intelligence (AI)?

At Rutgers' Art and Artificial Intelligence Laboratory, my colleagues and I proposed a novel algorithm that assessed the creativity of any given painting, while taking into account the painting’s context within the scope of art history.

In the end, we found that, when introduced with a large collection of works, the algorithm can successfully highlight paintings that art historians consider masterpieces of the medium.

The results show that humans are no longer the only judges of creativity. Computers can perform the same task – and may even be more objective.

How is creativity defined?

Of course, the algorithm depended on addressing a central question: how do you define – and measure – creativity?

There is a historically long and ongoing debate about how to define creativity. We can describe a person (a poet or a CEO), a product (a sculpture or a novel) or an idea as being “creative.”

In our work, we focused on the creativity of products. In doing so, we used the most common definition for creativity, which emphasizes the originality of the product, along with its lasting influence.

These criteria resonate with Kant’s definition of artistic genius, which emphasizes two conditions: being original and “exemplary.”

They’re also consistent with contemporary definitions, such as Margaret A Boden’s widely accepted notion of Historical Creativity (H-Creativity) and Personal/Psychological Creativity (P-Creativity). The former assesses the novelty and utility of the work with respect to scope of human history, while the latter evaluates the novelty of ideas with respect to its creator.

Building the algorithm

Using computer vision, we built a network of paintings from the 15th to 20th centuries. Using this web (or network) of paintings, we were able to make inferences about the originality and influence of each individual work.

Through a series of mathematical transformations, we showed that the problem quantifying creativity could be reduced to a variant of network centrality problems – a class of algorithms that are widely used in the analysis of social interaction, epidemic analysis and web searches. For example, when you search the web using Google, Google uses an algorithm of this type to navigate the vast network of pages to identify the individual pages that are most relevant to your search.

Any algorithm’s output depends on its input and parameter settings. In our case, the input was what the algorithm saw in the paintings: color, texture, use of perspective and subject matter. Our parameter setting was the definition of creativity: originality and lasting influence.

The algorithm made its conclusions without any encoded knowledge about art or art history, and made its assessments of paintings strictly by using visual analysis and considering their dates.

Innovation identified

Edvard Munch would be delighted to learn that the algorithm gave The Scream a high score.
Wikimedia Commons

When we ran an analysis of 1,700 paintings, there were several notable findings. For example, the algorithm scored the creativity of Edvard Munch’s The Scream (1893) much higher than its late 19th-century counterparts. This, of course, makes sense: it’s been deemed one of the most outstanding Expressionist paintings, and is one of the most-reproduced paintings of the 20th century.

The algorithm also gave Picasso’s Ladies of Avignon (1907) the highest creativity score of all the paintings it analyzed between 1904 and 1911. This is in line with the thinking of art historians, who have indicated that the painting’s flat picture plane and its application of Primitivism made it a highly innovative work of art – a direct precursor to Picasso’s Cubist style.

The algorithm pointed to several of Kazimir Malevich’s first Suprematism paintings that appeared in 1915 (such as Red Square) as highly creative as well. Its style was an outlier in a period then-dominated by Cubism. For the period between 1916 and 1945, the majority of the top-scoring paintings were by Piet Mondrian and Georgia O'Keeffe.

Of course, the algorithm didn’t always coincide with the general consensus among art historians.

For example, the algorithm gave a much higher score to Domenico Ghirlandaio’s Last Supper (1476) than to Leonardo da Vinci’s eponymous masterpiece, which appeared about 20 years later. The algorithm favored da Vinci’s St John the Baptist (1515) over his other religious paintings that it analyzed. Interestingly, da Vinci’s Mona Lisa didn’t score high by the algorithm.

A graph highlighting certain paintings deemed most creative by the algorithm.
Author provided

Withstanding the test of time

Given the aforementioned departures from the consensus of art historians (notably, the algorithm’s evaluation of da Vinci’s works), how do we know that the algorithm generally worked?

As a test, we conducted what we called “time machine experiments,” in which we changed the date of an artwork to some point in the past or in the future, and recomputed their creativity scores.

We found that paintings from the Impressionist, Post-Impressionist, Expressionist and Cubism movements saw significant gains in their creativity scores when moved back to around AD 1600. In contrast, Neoclassical paintings did not gain much when moved back to 1600, which is understandable, because Neoclassicism is considered a revival of the Renaissance.

Meanwhile, paintings from Renaissance and Baroque styles experienced losses in their creativity scores when moved forward to AD 1900.

We don’t want our research to be perceived as a potential replacement for art historians, nor do we hold the opinion that computers are a better determinant of a work’s value than a set of human eyes.

Rather, we’re motivated by Artificial Intelligence (AI). The ultimate goal of research in AI is to make machines that have perceptual, cognitive and intellectual abilities similar to those of humans.

We believe that judging creativity is a challenging task that combines these three abilities, and our results are an important breakthrough: proof that a machine can perceive, visually analyze and consider paintings much like humans can.

This article was originally published on The Conversation. Read the original article.

Related article:

Pricing Genius

by David W. GalensonAdded 30.07.2015
Art world sophisticates often casually dismiss the market for art as irrational and arbitrary. The critic Robert Hughes memorably declared in 1978 that "The price of a work of art is an index of pure, irrational desire." More recently, in 2006,...

Browse articles by author

More Essays

Jul 31st 2021
EXTRACT: "If we want to live in a world that is good for pollinators, as well as the rest of us, big changes are needed in our environment, and our food system. This is why many beekeepers change their diet and their shopping, eating more locally grown vegetables that aren’t treated with pesticides. ...... Being willing to buy fruit and vegetables that may have the occasional insect living in it is better for us and for nature. To live more harmoniously with the natural world, we need to relax about larvae in the lettuce and slugs in the spinach."
Jul 22nd 2021
EXTRACT: "You’d think our brush with mortality through the pandemic would have brought some of this home to us. You’d think it would give us pause for thought about what really matters to us: the kind of world we want for our children; the kind of society we want to live in. And for many people it has. In a survey carried out during lockdown in the UK, 85% of respondents found something in their changed conditions they felt worth keeping and fewer than 10% wanted a complete return to normal."
Jul 20th 2021
EXTRACT: "English artist Damien Hirst’s latest project, “The Currency”, is an artwork in two forms. Its physical form is 10,000 unique hand-painted A4 sheets covered in colourful dots. In the same way as paper money, each sheet includes a holographic image of Hirst, a signature, a microdot and – in place of a serial number – a small individual message. The second part of the artwork is that each of these hand-painted sheets has a corresponding NFT (non-fungible token). NFTs are digital certificates of ownership which exist on the secure online ledgers that are known as blockchains. ---- The way that “The Currency” works is that collectors will not be buying the physical artwork immediately. Instead, they will pay US$2,000 (£1,458) for the NFT and then have a year to decide whether they want the digital or the physical version. Once the collector selects one, the other will be destroyed. ---- So what is going on here, and what does it tell us about art and money?"
Jul 20th 2021
EXTRACT: "Ellison was an abstract expressionist painter, who, having come to New York City from West Texas in 1962, was as he said “unable to find traction” as a painter. At the same time, he began collecting ceramic objects and educating himself about this field of art as he went along. In 2009 he bestowed on the Metropolitan Museum of Art over 300 extraordinary examples of American ceramics, spanning the years 1876 through 1956. Since then, Ellison has gifted to the Museum over 600 works – including a significant collection of European art pottery in 2013, and most recently over 125 modern and contemporary clay vessels and objects – making the Museum one of the most significant repositories of Art Pottery in the world. ---- The current exhibition presents nearly 80 pieces drawn from Ellison’s latest donation, and it is a thoroughly captivating show; even where (or perhaps especially where) the works are outlandish, bizarre, sometimes almost monstrous, but nonetheless enthralling."
Jul 11th 2021
EXTRACT: "Over the course of England’s journey to the Euro 2020 final, one of the most fascinating plays has been happening just off the pitch. Whenever the TV camera cuts to the team’s manager Gareth Southgate, he is occasionally seen standing alone on the edge of the field, urging his team on. ---- But most of the time he is deep in conversation with his assistant Steve Holland. ---- A recent study of English football culture points to a shift away from what the authors term “Beckhamisation”, after the former England captain and Manchester United star player David Beckham – a popular and instantly recognisable symbol of that period of football history (though, it is not suggested the culture was his creation). ---- During the 1990s, the study claims, this “Beckhamisation” saw high octane management practices imported from the corporate world into football. ---- In recent years, this has been replaced by “Southgatism”, a leadership style which that study describes as “modest, self-deprecating, down to earth, diverse and progressive”. "
Jun 30th 2021
EXTRACT: "New York’s Museum of Modern Art is currently presenting an exhibition devoted to an in-depth review of Paul Cézanne’s drawings. If there is any criticism to be made of this extraordinary show, it is that it is frankly overwhelming: with roughly 280 pencil, ink and gouache drawings and watercolors (and even a handful of oil paintings), there is so much to take in that two or three visits to the exhibition may be required to do it justice."
Jun 25th 2021
EXTRACT: "Cognitive flexibility provides us with the ability to see that what we are doing is not leading to success and to make the appropriate changes to achieve it." .... "Flexible thinking is key to creativity – in other words, the ability to think of new ideas, make novel connections between ideas, and make new inventions." .... "The good news is that it seems you can train cognitive flexibility."
Jun 17th 2021
EXTRACT: "Confronting our complex history and ultimately embracing a more equitable, balanced, and humble culture may be a tall order in these fractious times. But that makes it even more imperative that we fully reckon with who we are and who we are capable of becoming."
Jun 11th 2021
EXTARCT: "A further health benefit of hiking is that it’s classed as “green exercise”. This refers to the added health benefit that doing physical activity in nature has on us. Research shows that not only can green exercise decrease blood pressure, it also benefits mental wellbeing by improving mood and reducing depression to a greater extent than exercising indoors can."
Jun 10th 2021
EXTRACT: "“The greatness of a nation and its moral progress,” Mahatma Gandhi said, “can be judged by the way its animals are treated.” If we apply that test to the world as a whole, how much moral progress have we made over the past two millennia? ...... That question is suggested by The Golden Ass, arguably the world’s earliest surviving novel, written around 170 CE, when Emperor Marcus Aurelius ruled the Roman Empire. Apuleius, the author, was an African philosopher and writer, born in what is now the Algerian city of M’Daourouch."
Jun 4th 2021
EXTRACT: "Research we’ve done, which looked at 37 adults with type 2 diabetes, found that over two weeks, prolonged sitting was associated with high blood sugar levels. But we also found that when people stood up or walked around between periods of sitting, they had lower blood sugar levels. Other studies have also had similar results."
May 28th 2021
EXTRACT: "Paul Van Doren's legacy lies in a famous company, and in his advice to young entrepreneurs to get their hands dirty, and to know what goes into making what they are selling."
May 19th 2021
EXTRACT: "May 7th marked three hundred and ten years since the philosopher David Hume was born. He is chiefly remembered as the most original and destructive of the early modern empiricists, following John Locke and George Berkeley." .... " Shocking as it may (and should) sound, Hume is implying nothing less than that the next time you turn the key in your car ignition, you are as justified to expect the engine will start as you are in believing it will turn into a pumpkin. For there is a radical contingency that pervades all our experience. We could wake up tomorrow to a world that looks and behaves very differently to the one we are in now. Matters of fact are dependent on experience and can never be known a priori — they are purely contingent, and could always turn out different than what we expect."
May 1st 2021
EXTRACT: " The sad reality is that the Mizrahim (Jews of Middle Eastern and North African descent) were discriminated against from the day of Israel’s inception, whose Ashkenazi (European Jewish) leaders viewed them as intellectually inferior, “backward,” and “too Arab,” and treated them as such, largely because the Ashkenazim agenda was to maintain their upper-class status while controlling the levers of power, which remain prevalent to this day." ..... " The greatest heartbreaking outcome is that for yet another generation of Israelis, growing up in these debilitating conditions has a direct effect on their cognitive development. A 2015 study published in Nature Neuroscience found that “family income is significantly correlated with children’s brain size…increases in income were associated with the greatest increases in brain surface area among the poorest children.” "
Apr 25th 2021
EXTRACT: "We all owe Farah Nabulsi an enormous debt of gratitude. In a short 24-minute film, The Present, she has exposed the oppressive indecency of the Israeli occupation while telling the deeply moving story of a Palestinian family. What is especially exciting is that after winning awards at a number of international film festivals​, Ms. Nabulsi has been nominated for an Academy Award for this remarkable work of art. " 
Apr 25th 2021
EXTRACT: "When I crashed to the floor of my home in Bordeaux recently after two months of Covid-19 dizziness, I was annoyed. The next day I collapsed again. Now I was worried. What I didn’t know was that my brain was sloshing around inside my skull, causing a mild concussion. Nor did I know that I was in for a whole new world of weird and wonderful hallucinations."
Apr 13th 2021
EXTRACT: "Overall, our review has found that there isn’t evidence to back up the claims that veganism is good for your heart. But that is partly because there are few studies ....... But veganism may have other health benefits. Vegans have been found to have a healthier weight and lower blood glucose levels than those who consume meat and dairy. They are also less likely to develop cancer, high blood pressure and diabetes. "
Apr 8th 2021
EXTRACT: "Pollock’s universe, the universe of Mural, cannot be said to be a rational universe. Nor is it simply devoid of all sense. It is not a purely imaginary world, although in it everything is in a constant state of flux. Mural invokes one of the oldest questions of philosophy, a question going back to the Pre-Socratic philosophers Parmenides and Heraclitus – namely, whether the nature of Reality constitutes unchanging permanence or constant movement and flux. For Pollock, the only thing that is truly unchanging is change itself. The only certainty is that all is uncertain."
Apr 8th 2021
EXTRACT: "Many present day politicians appear to have psychopathic and narcissistic traits too. It’s easy to spot such leaders, because they are always authoritarian, following hardline policies. They try to subvert democracy, to reduce the freedom of the press and clamp down on dissent. They are obsessed with national prestige, and often persecute minority groups. And they are always corrupt and lacking in moral principles."
Apr 6th 2021
EXTRACT: "This has led some to claim that not just half, but perhaps nearly all advertising money is wasted, at least online. There are similar results outside of commerce. One review of field experiments in political campaigning argued “the best estimate of the effects of campaign contact and advertising on Americans’ candidates choices in general elections is zero”. Zero!"