Feasibility of facial expression analysis
Background
In children, palatability is crucial for ensuring patient acceptability and treatment compliance for orally administered medicines. Understanding children’s taste sensitivity and preferences can help formulators develop more acceptable paediatric medicines. Furthermore, the collection of data in a home environment places less strain on participants and allows for natural behaviour.
Challenge
We investigate whether using computer vision and machine learning techniques on videos of children reacting to gustatory taste strips can provide an objective evaluation of palatability.
Solution
Primary school children, aged 5-11 years, tasted four different flavoured strips: no taste, bitter, sweet and sour (UCL REC 4612/029). Data was collected at home, under the supervision of a guardian. Reactions were recorded and uploaded using the Aparito Atom5™ app and a smartphone camera. Participants also rated each taste strip on a 5-point hedonic scale. To analyse the changes in the children’s facial expressions in reaction to tasting the strips, we use a machine learning framework for pose estimation, MediaPipe (MP). Then, using a comprehensive data-driven process we analyse and classify the reaction of children to different tastes using a baseline and best reaction frame from the videos that capture their facial expressions.
Outcomes
A total of 215 videos and 252 self-reported scores from 64 participants were received. Children’s ratings from the hedonic scale showed expected results: children like sweetness, dislike bitterness and have varying opinions for sourness.
We observed a wide facial variation across participants in the magnitude, onset and duration of reactions.
Challenges resulting from home-recorded videos are lack of standardisation and inability to provide timely feedback. Moreover, another challenge is to compare facial measurements (brow elevation, mouth openness, eye openness, etc.) extracted from videos of different tastings, whilst accounting for the variations over time in the face’s position and orientation.
We explored different methods for rescaling and transforming the extracted measurements, to overcome this challenge. The rescaled measurements are used to train machine learning classifiers that attempt to categorise the different tastes and the hedonic ratings.
The ability to objectively measure how children feel about the taste of medicines has great
potential in helping find the most palatable formulation.
This study demonstrated the feasibility of collecting such data in a decentralised, at-home way. Ultimately, this approach to palatability assessment can improve the evaluation of paediatric taste specificities, thus making paediatric medicines more acceptable.