10. Classifying Constable’s Clouds¶

The V&A and Science Museum have many items in common in their collections, but often collected for different reasons, some of the objects showing how people have responded to some of the problems and challenges of their times. Living in the United Kingdom, one of the most recurring problems we want an answer to is, what kind of rainy clouds might be hanging over our heads today? And equally importantly (for the sake of this example), what clouds were hanging over Constable’s head 200 years ago?. The Science Museum has, on loan from the Royal Meteorological Society, a set of sketches classifying and naming clouds developed by the pioneering meteorologist Luke Howard. The V&A holds many of Constable’s drawings and sketches. So, let’s see if we can use the Science Museum images to identify the Constable’s clouds.

import sys
sys.path.append("/srv/explore-the-collections/code/ivpy/src")

Note

The proper study of weather in Constable paintings and the relationship to Howard’s cloud studies has been carried out by researchers in various disciplines; this is not one of these serious studies and no art historical or meterological conclusions should be drawn from this artifical example. Those interested in using historical sources for identifying patterns in weather may like to look at the Old Weather project.

Luke Howard Cloud Classification¶

As discussed by the Science Museum, Howard wanted to provide names for each different type of cloud. He documented his observations with a set of cloud sketches, viewable in the Science Museum collection. So first we need to retrieve these via the Science Museum API.

import requests
import pandas as pd

req = requests.get("https://collection.sciencemuseumgroup.org.uk/search/images/makers/luke-howard?q=cloud%20study&page%5Bsize%5D=18", headers = {"Accept": "application/json"})
sciencemuseum_clouds_json = req.json()
sciencemuseum_clouds = []
for obj in sciencemuseum_clouds_json['data']:
    sciencemuseum_clouds.append([obj['attributes']['description'][0]['value'], obj['attributes']['multimedia'][0]['processed']['large_thumbnail']['location'], obj['links']['self']])

sciencemuseum_clouds_df = pd.DataFrame(sciencemuseum_clouds, columns=['Title', 'Thumbnail', 'Link'])
sciencemuseum_clouds_df.head(10)

	Title	Thumbnail	Link
0	Cloud study by Luke Howard, c1803-1811: Stratu...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
1	Cloud study by Luke Howard, c1803-1811: Possib...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
2	Cloud study by Luke Howard, c1803-1811: Cumulu...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
3	Cloud study by Luke Howard, c1803-1811: Cirrus...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
4	Cloud study by Luke Howard, c1803-1811: Stratu...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
5	Cloud study by Luke Howard, c1803-1811: Nimbus...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
6	Cloud study by Luke Howard, 1811: Light cirros...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
7	Cloud study by Luke Howard, c1803-1811: Cirroc...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
8	Cloud study by Luke Howard, c1803-1811: Cumulu...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...
9	Cloud study by Luke Howard, c1803-1811: Cirroc...	https://coimages.sciencemuseumgroup.org.uk/ima...	https://collection.sciencemuseumgroup.org.uk/o...

from ivpy import attach,show

attach(sciencemuseum_clouds_df, "Thumbnail")
show()

All suitably cloudlike. Some of the sketches contain landscape scenery which will be problematic and should be excluded if this was a real attempt to construct a cloud classification training set (and the many other steps needed to construct a real training set, such os the removal of signatures/text, removing the backdrop, and so on)

So now we have what we need for our small classification training set, lets start looking at some Constable clouds at the V&A using our API.

Constable Clouds¶

import requests
import pandas as pd

vam_clouds_df = pd.read_csv("https://api.vam.ac.uk/v2/objects/search?q_object_title=Cloud&id_person=A8267&images_exist=1&response_format=csv")
vam_clouds_df.head(5)

	accessionNumber	accessionYear	systemNumber	objectType	_primaryTitle	_primaryPlace	_primaryMaker__name	_primaryMaker__association	_primaryDate	_primaryImageId	_sampleMaterial	_sampleTechnique	_sampleStyle	_currentLocation__displayName	_objectContentWarning	_imageContentWarning
0	590-1888	1888	O82726	Oil painting	Study of Clouds	Great Britain	Constable, John (RA)	artist	05/09/1822	2014HA9998	oil paint	oil painting	British School	Prints & Drawings Study Room, room WS	False	False
1	162-1888	1888	O25338	Oil painting	Study of clouds and trees	London	Constable, John	painters (artists)	1821-1822	2006AP1993	oil paint	oil painting	British School	Prints & Drawings Study Room, room WS	False	False
2	784-1888	1888	O82649	Oil painting	Study of Cirrus Clouds	Great Britain	Constable, John (RA)	artist	ca. 1822	2014GX8500	NaN	oil painting	British School	Prints & Drawings Study Room, room WS	False	False
3	P.8-1973	1973	O1041774	Watercolour	Landscape with trees and clouds	England	Constable	artist	ca. 1820	2013GE8594	pencil	watercolour drawing	NaN	Prints & Drawings Study Room, level H	False	False
4	240-1888	1888	O125166	Watercolour	Study of clouds above a wide landscape	great britain	Constable, John (RA)	artist	15/09/1830	2006BF8141	pencil	NaN	British School	Prints & Drawings Study Room, level H	False	False

IIIF_IMAGE_URL = "https://framemark.vam.ac.uk/collections/%s/full/!100,100/0/default.jpg"
vam_clouds_df._primaryImageId = [IIIF_IMAGE_URL % item for item in vam_clouds_df._primaryImageId]
attach(vam_clouds_df, "_primaryImageId")
show()

So, a few cloud studies, one helpfully titled ‘Study of Cirrus Clouds’ which seems a good one to start with.

Perceptual Hashing¶

One way to measure the distance between two images is the use of Perceptual Hashing. Let’s use the ImageHash library for Python to see how close the images are by the Perceptual Hashing algorithm which, at a very simplified explanation, subtracts one images representation from another one and gives a distance measure of how similiar the two images are.

First we need to let ImageHash analysis each image and store the representation of them from each collection.

import imagehash
from PIL import Image
import requests

vam_cloud_hashes = []
for index, obj in vam_clouds_df.iterrows():
  hash = imagehash.phash(Image.open(requests.get(obj['_primaryImageId'], stream=True).raw))
  vam_cloud_hashes.append(hash)
    
vam_clouds_df['Hash'] = vam_cloud_hashes

sciencemuseum_cloud_hashes = []
for index, obj in sciencemuseum_clouds_df.iterrows():
  hash = imagehash.phash(Image.open(requests.get(obj['Thumbnail'], stream=True).raw))
  sciencemuseum_cloud_hashes.append(hash)
    
sciencemuseum_clouds_df['Hash'] = sciencemuseum_cloud_hashes

Now we have the two hashes, we can subtract one from the other to see the distance between the images (the smaller the distance, the more similiar the images). Let’s look at the distance from each of the Science Museum images to Constable’s ‘Study of Cirrus Clouds’ sketch.

sciencemuseum_clouds_df['Hash'] - vam_clouds_df['Hash'].iloc[2]

   36
   34
   28
   24
   28
   32
   28
   32
   32
   32
  28
  32
  28
  32
  28
  34
  36
  32
Name: Hash, dtype: object

This has found that the most similiar (with the lowest distance of 24 using perceptual hashing) between Constable’s ‘Study of Cirrus Clouds’ is Luke Howards Cloud Study of Cirrus.

So far then (somewhat astonishingly given the total lack of real validity in this example) this seems to be working. Let’s try now with a second Constable sketch, this time without a helpful title indicating the type of clouds.

sciencemuseum_clouds_df['Hash'] - vam_clouds_df['Hash'].iloc[0]

   34
   30
   30
   28
   28
   30
   30
   38
   28
   24
  24
  32
  22
  36
  32
  38
  36
  28
Name: Hash, dtype: object

So, by this very very unscientific measure, the 13th sketch by Howard is the closest match to Constable’s Study of Clouds, which is Cloud Study of Dark Cirrostratus.

Further Work¶

An obvious further step in a real examination would be to apply some more recent image recognition techniques involving machine learning and neural network, and using a larger training set of cloud classifications. Let us know if you try this out.

Data Explorations

10. Classifying Constable’s Clouds¶

Luke Howard Cloud Classification¶

Constable Clouds¶

Perceptual Hashing¶

Further Work¶