The first step in emotion detection AI is to actually take the image frame from a camera feed and detect the human face. In this process the face is located and highlighted with a box. In this little box coordinates are used to pinpoint the exact face location in real time. This is where facial recognition AI comes into play. However, sometimes this step fails if the image or video is not clear enough to detect a face. This can be due to difficult lighting conditions, unusual head positions, distance, and other obstructions.
The next step is to crop, resize, and rotate as much as necessary. This step is also known as preprocessing the image. After the face is detected the picture is enhanced in order to guarantee “accurate” emotion detection. Some changes that the picture might go through are image smoothing and even color corrections. This makes it possible to improve the chances of getting correct analysis.
After preparing the image it is finally ready to have emotions be extracted from them. Facial features are then deeply analyzed in order to be categorized into the 7 possible emotions such as happiness, sadness, fear, disgust, surprise, anger, or neutral. This is done by using these AIs to study the “motion of facial landmarks, distances between facial landmarks, gradient features, facial texture, and more.” This has to do with the separate classification of muscle contraction that happens in your face when having a positive or negative response to something.
However, before this classification process is possible these algorithms are trained to recognize previous things they’ve seen before. In order for these algorithms to make connections to what a face might look like when portraying a specific emotion they must have prior knowledge of what these emotions actually look like. For example, if the AI is shown many different pictures of what a “happy face” looks like then in the future it is able to distinguish a happy face when shown one. This is how these AIs are able to identify and label what emotion is portrayed accordingly by matching them to what it’s seen before.
“Emotion detection technology requires two techniques: computer vision, to precisely identify facial expressions, and machine learning algorithms to analyze and interpret the emotional content of those facial features.”
– Oscar Schwartz from The Guardian article
Emotion Detection AI in Cameras 