6. Animal, Vegetable or ML¶
Animal, Vegetable or Mineral is a basic form of classification that has a long history, and inevitably in the UK, a television panel show that has featured the V&A collections, as can be seen in this recording from the BBC television show of the same name from 1958. In that show the panellists were challenged to identify the following objects:
Peruvian Doll
Acupuncture Doll (from the Royal College of Surgeons Collection
Axe Sheath
Let’s see if some (very simple) machine learning can do better than the humans on identifying the objects featured in the show, using the following applications and libraries:
Image Classification¶
First we need to build the model, using a training dataset from CIFAR-100. This has 10 broad classifications and 100 more detailed ones within those.
Note
This step will take some time, and ideally should be run for a much longer time than carried out here to get better results. This example is not a real indication of the quality of human or machine learning. For a real introduction to machine learning, see the many guides on the internet for this (perhaps start with the Keras website.
Search: Running Trial #1
Hyperparameter |Value |Best Value So Far
image_block_1/b...|vanilla |?
image_block_1/n...|True |?
image_block_1/a...|False |?
image_block_1/c...|3 |?
image_block_1/c...|1 |?
image_block_1/c...|2 |?
image_block_1/c...|True |?
image_block_1/c...|False |?
image_block_1/c...|0.25 |?
image_block_1/c...|32 |?
image_block_1/c...|64 |?
classification_...|flatten |?
classification_...|0.5 |?
optimizer |adam |?
learning_rate |0.001 |?
The classifications in the CIFAR100 training set are as follows:
cifar100_broad_classes = [
"beaver", "dolphin", "otter", "seal", "whale",
"aquarium fish", "flatfish", "ray", "shark", "trout",
"orchids", "poppies", "roses", "sunflowers", "tulips",
"bottles", "bowls", "cans", "cups", "plates",
"apples", "mushrooms", "oranges", "pears", "sweet peppers",
"clock", "computer keyboard", "lamp", "telephone", "television",
"bed", "chair", "couch", "table", "wardrobe",
"bee", "beetle", "butterfly", "caterpillar", "cockroach",
"bear", "leopard", "lion", "tiger", "wolf",
"bridge", "castle", "house", "road", "skyscraper",
"cloud", "forest", "mountain", "plain", "sea",
"camel", "cattle", "chimpanzee", "elephant", "kangaroo",
"fox", "porcupine", "possum", "raccoon", "skunk",
"crab", "lobster", "snail", "spider", "worm",
"baby", "boy", "girl", "man", "woman",
"crocodile", "dinosaur", "lizard", "snake", "turtle",
"hamster", "mouse", "rabbit", "shrew", "squirrel",
"maple", "oak", "palm", "pine", "willow",
"bicycle", "bus", "motorcycle", "pickup truck", "train",
"lawn-mower", "rocket", "streetcar", "tank", "tractor"]
So, it looks the only one we can try this with is the ‘Hanukkah lamp’. This is yet another indication that this is not a very suitable test of machine learning in this example, and a training set more designed for cultural heritage object classification would perform much better (perhaps someone is working on this training set somewhere right now).
Anyway, let’s see what we can get from it. First we need to convert our object image into the right matrix size (32 by 32 pixels) to match the neural network inputs.
import requests
from PIL import Image
req = requests.get("https://api.vam.ac.uk/v2/object/O72424")
lamp = req.json()
lamp_thumbnail_url = lamp['meta']['images']['_primary_thumbnail']
response = requests.get(lamp_thumbnail_url, stream=True)
img_array = None
img = Image.open(response.raw)
img = img.convert('RGB')
img = img.resize((32,32), Image.NEAREST)
img_array = image.img_to_array(img)
Prediction¶
Now we have the image at the right size to test, we can feed it into the classifier trained on the CIFAR-100 training data.
WARNING:tensorflow:6 out of the last 11 calls to <function Model.make_predict_function.<locals>.predict_function at 0x7f537c4e4b80> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
Object has class television
Hmmm, well, not quite right , but at least it’s broadly in the right area (‘household electrical items’). Let’s take a look at what the algorithm was seeing:
Of course, the training set’s idea of a lamp is something rather different than this, so a television isn’t such an unusual guess. It’s likely more model training would not really affect the predicitions accuracy, as the key issue here is the training data not really being applicable to rare cultural heritage objects (a huge clue to that being in the broad group name of ‘household electrical items’ being applied to a 16th century lamp). But with enough examples of objects from multiple collections in a cultural heritage training dataset, there could be some improvements in the predictions for some of the items featured on the TV show.
Victory for Humans¶
For now…