AI used for recognizing emojis on wallpaper

Question

I want to build an app that will recognize what emojis have been used on the wallpaper.

So for instance this app will receive an input image like [this](https://i.stack.imgur.com/5wRQm.png)

And on output should return an array of names of recognized emojis:
```
[
  "Smiling Face with Sunglasses", 
  "Grinning Face with Smiling Eyes", 
  "Kissing Face with Closed Eyes"
]
```

Of course, the names of these emojis will come from the names of files of training images.
For example, [this](https://i.stack.imgur.com/BaEGG.png) file, will be called `Grinning_Face_with_Smiling_Eyes.jpg`

I would like to use `AWS Rekognition Label`, but they require a minimum of 10 images of each emoji for training. 
As you know, I can only provide one image of each emoji, because there is no more option, they are in 2D ;)

Now my question is:
What should I do? How can I skip these requirements? Which service should I choose?
On the stack, I have read that I should for instance rotate each image 12 times by 30o, or crop the emoji by half. I have done that, but, precision is very small - around `0.3`

PS. In real business instead of emojis, there are covers of the books, which AI has to recognize. There is also one image per book-cover photo in 2D.

Answer

Rotation is a good start, but you might want to go for more robust transformations and a more robust model based on fine-tuning. It really depends on what you plan to feed it later - how much variability there will be in the images you expect to do inference on.

This blog talks about transfer learning / fine-tuning and could be helpful. https://aws.amazon.com/blogs/machine-learning/run-image-classification-with-amazon-sagemaker-jumpstart/

Then take a look at the methods described here for data augmentation: https://www.tensorflow.org/tutorials/images/data_augmentation

Good luck!

Answer

You have to train an AI with the same material it'll be used to classify, not something close. In this case you need to train it with "emoji on wallpaper" not just "emoji"

So you'll need a library of images that you'd normally classify with the AI, that a human has manually classified, and use those to train it.

AI used for recognizing emojis on wallpaper

Relevant content