Feasibility of facial expression analysis

Background

In children, palatability is crucial for ensuring patient acceptability and treatment compliance for orally administered medicines. Understanding children’s taste sensitivity and preferences can help formulators develop more acceptable paediatric medicines. Furthermore, the collection of data in a home environment places less strain on participants and allows for natural behaviour.

Challenge

We investigate whether using computer vision and machine learning techniques on videos of children reacting to gustatory taste strips can provide an objective evaluation of palatability.

Solution

Primary school children, aged 5-11 years, tasted four different flavoured strips: no taste, bitter, sweet and sour (UCL REC 4612/029). Data was collected at home, under the supervision of a guardian. Reactions were recorded and uploaded using the Aparito Atom5™ app and a smartphone camera. Participants also rated each taste strip on a 5-point hedonic scale. To analyse the changes in the children’s facial expressions in reaction to tasting the strips, we use a machine learning framework for pose estimation, MediaPipe (MP). Then, using a comprehensive data-driven process we analyse and classify the reaction of children to different tastes using a baseline and best reaction frame from the videos that capture their facial expressions.

Outcomes

A total of 215 videos and 252 self-reported scores from 64 participants were received. Children’s ratings from the hedonic scale showed expected results: children like sweetness, dislike bitterness and have varying opinions for sourness.

We observed a wide facial variation across participants in the magnitude, onset and duration of reactions.

Challenges resulting from home-recorded videos are lack of standardisation and inability to provide timely feedback. Moreover, another challenge is to compare facial measurements (brow elevation, mouth openness, eye openness, etc.) extracted from videos of different tastings, whilst accounting for the variations over time in the face’s position and orientation.

We explored different methods for rescaling and transforming the extracted measurements, to overcome this challenge. The rescaled measurements are used to train machine learning classifiers that attempt to categorise the different tastes and the hedonic ratings.

The ability to objectively measure how children feel about the taste of medicines has great
potential in helping find the most palatable formulation.

This study demonstrated the feasibility of collecting such data in a decentralised, at-home way. Ultimately, this approach to palatability assessment can improve the evaluation of paediatric taste specificities, thus making paediatric medicines more acceptable.

Feasibility of facial expression analysis as an objective palatability assessment of paediatric medicine poster
View the poster presentation TASTY: Feasibility of facial expression analysis as an objective palatability assessment of paediatric medicine

Our Credentials

21 CFR Part 11 compliant
Crest
Cyber Essentials Plus
ePrivacy App
ISO 13485
ISO 27001

More Success Stories

Success Story:
Feasibility of facial expression analysis

20/07/2022

Success Story:
 ePROs for Traumatic Brain Injury Patients

19/07/2022

Success Story:
Digital Tools for Outcome Measures in Gaucher Disease

28/03/2022

Success Story:
Remote Patient Monitoring for Oncology Patients

Success Story:
ePROs and eConsent for Long COVID (TLC)

17/03/2022

Read More
The University of Birmingham Clocktower

 ePROs for Traumatic Brain Injury Patients

With The University of Birmingham and the National Institute for Health Research (NIHR).

Background

Each year, 1.4 million people attend A&E in England with a Traumatic Brain Injury (TBI) & over 50 million people worldwide have sustained a TBI. Advances in critical care, imaging and the reorganisation of trauma health systems mean that more people live with the damage caused by the TBI for longer.

Challenge

This study aimed to develop and assess the feasibility of an Electronic Patient-Reported Outcomes (ePRO) system for inclusion within routine clinical care & TBI research; in this instance, getting people with a TBI to report their symptoms electronically utilising questionnaires via the Aparito Atom5™ platform.

Solution

First was a qualitative study to obtain the views of PROs and ePROs from people with a TBI, carers and health care providers, followed by a usability study.

Applying the results from the qualitative study and feedback from the PPI group, Aparito deployed Atom5™ to collect ePROM responses via a patient-facing app and a web-based clinician dashboard.

Outcomes

Patient attitudes towards ePROs were overwhelmingly positive:

  • Less burdensome for patients & clinicians
  • Fewer data entry errors
  • Easy real-time data/remote monitoring & response
  • Ability to send/receive feedback easily

Aparito’s Atom5™ platform enabled the team at the University of Birmingham to demonstrate the potential to capture PROs electronically in routine clinical practice and TBI research.

Our Credentials

21 CFR Part 11 compliant
Crest
Cyber Essentials Plus
ePrivacy App
ISO 13485
ISO 27001
Read More

Digital Tools for Outcome Measures in Gaucher Disease

Background

Gaucher Disease (GD) is an inherited lysosomal storage disorder. There are 3 subtypes;

  • Type 1 has no neurological involvement (and is treated with enzyme replacement therapy);
  • Type 2 results in infant death; 
  • Type 3 is a heterogeneous disorder characterised by progressive neurological decline throughout childhood and adult life. 

Clinical features of neurological GD (nGD) include specific saccadic eye movement defect, altered muscle tone, coordination impairment, tremor and late in the disease, ataxia. Patients also have varying severity of bone disease, kyphosis, scoliosis, hearing impairment and other non-neurological features such as lung infiltration or cardiac disease.

Challenge

Endeavours to find a therapy to modify nGD are limited by a lack of meaningful clinical outcome measures which are acceptable to patients. Disease severity in nGD may be described by modified Severity Scoring Tool (mSST) which, although useful, fails to account for the functional impact of disease on patients and only give a momentary account of function, overlooking disease fluctuations and the factors which provoke them.

Solution

Twenty-one patients were enrolled in the UK and participated for up to 12 months. Clinical measures included a neurological examination, the mSST, 6-Minute Walk Test (6MWT) and GAITIRite or Zeno Walkway gait analysis. Atom5TM was paired with a 3D accelerometer device worn on the wrist, which captured data in 30-minute epochs. The paired app pushed out Patient-Reported Outcomes (PROs) and Quality of Life (QoL) scales at pre-set intervals. Patients could also record visits to healthcare professionals, other ‘events’ e.g. falls, seizures, etc, and were encouraged to record sleep quality.

Outcomes

Five patients with Type 1 GD and 16 patients with nGD were included. 

Fifteen patients completed the 6MWT; the mean distance walked by nGD patients was 391m and by Type 1 patients was 475.67m. There was no statistical correlation between disease severity on mSST and 6MWT distance. 

Wearable device data comprised 3 different variables (average daily maximum (ADM), average daily steps (ADS), and average daily steps per 30-minute epoch(ADE)). These were considerably higher in the GD1 group compared with the nGD group (ADM almost 3 times; ADS 2.5times; ADE almost double). 

Seven out of the 9 nGD patients reported bone pain as an event indicating that this is a significant disease feature across the cohort. 

The CHU9D showed a statistically significant difference between disease groups, nGD patients reporting overall lower health-related quality of life. Perceived stress was also significantly higher in patients with nGD than GD1 patients.

Wearable device data showed a much higher level of activity in GD1 compared with nGD patients. Bone symptoms in nGD patients have a greater functional impact on activity and quality of life than perhaps previously recognised.

Our Credentials

21 CFR Part 11 compliant
Crest
Cyber Essentials Plus
ePrivacy App
ISO 13485
ISO 27001
Read More

Remote Patient Monitoring for Oncology Patients

With Betsi Cadwaladr University Health Board and NHSx Techforce 19.

Background

At the onset of the COVID-19 pandemic, Aparito joined a study to provide remote patient monitoring for oncology patients in collaboration with Betsi Cadwaladr University Health Board and NHSx Techforce 19.

Challenge

Lockdown removed the ability of the Betsi Cadwaladr University Health Board to monitor oncology patients via inpatient clinics and they needed a means of remote patient monitoring to enable them to continue to collect patient data. 

The study sought to demonstrate the feasibility of multidimensional remote monitoring of cancer outpatients during the Covid-19 pandemic and evaluate the quality of the data collected and insight that can be retrieved from it (for more detail, read our publication, “Embracing Change: Learnings From Implementing Multidimensional Digital Remote Monitoring in Oncology Patients at a District General Hospital During the COVID-19 Pandemic“).

Solution

Cancer patients under active treatment or surveillance were invited by the clinical team during routine appointments (mostly conducted over the phone) or via the North Wales Patients Forum.

Remote monitoring and decentralisation allowed for rapid deployment and patient enrolment and study to take place when so many were suspended or delayed. 

High data capture was achieved via Atom5™ and good data quality allowed for insightful analyses to be performed informing on near real-time patient’s health.

Outcomes

The results for app usage and wearable engagement were impressive:

  • Over 2,800 patient days were collected via the Aparito Atom5™ app with a median engagement of 73%
  • 80% of the patients were recruited in just two weeks
  • Median engagement with the wearable device was an amazing 89% 

Aparito’s Atom5™ platform enabled patients taking part in chemotherapy to report their health status via a user-friendly interface in near real-time and provided hospital staff with the data to monitor patients at home.

Ready to get started?

Share your study requirements and unlock the potential of Atom5™ for your clinical trial!

Our Credentials

21 CFR Part 11 compliant
Crest
Cyber Essentials Plus
ePrivacy App
ISO 13485
ISO 27001

Our Success Stories

Success Story:
Feasibility of facial expression analysis

20/07/2022

Success Story:
 ePROs for Traumatic Brain Injury Patients

19/07/2022

Success Story:
Digital Tools for Outcome Measures in Gaucher Disease

28/03/2022

Success Story:
Remote Patient Monitoring for Oncology Patients

Success Story:
ePROs and eConsent for Long COVID (TLC)

17/03/2022

Read More
The University of Birmingham Clocktower

ePROs and eConsent for Long COVID (TLC)

The University of Birmingham, the National Institute for Health Research (NIHR) and UK Research and Innovation (UKRI). 

Background

Treatment & Understanding of Long COVID (TLC) is a two-year project pioneered by the University of Birmingham funded through the NIHR and UKRI that follows a UK-wide joint research call to fund ambitious and comprehensive Long COVID research. 

Challenge

The study required the development of a new ePRO measure, The Symptom Burden Questionnaire™ (SBQ™) for Long COVID to capture the unique symptoms of the disease. Developed by the team at the University of Birmingham, the SBQ™ asks patients to log their responses to a series of questions designed to understand symptom burden in adults with Long COVID. 

Solution

Centralising the study wasn’t feasible due to the commitment that patients would need to make and the requirement for frequent responses to the study’s SBQ™, so Aparito delivered the SBQ™ via the Atom5™ app’s ePRO module to provide critical support and information to empower patients in self-managing long COVID and report their symptoms.

To manage ongoing patient consent we deployed our eConsent module to provide a regulatory-compliant solution.  

We worked closely with national UK Long COVID support groups at each stage of the app’s development to ensure its acceptability, usability, and relevance to patients. This approach empowered Aparito to evolve and develop a new approach to monitoring Long COVID symptoms.

Outcomes

Aparito’s Atom5™ platform allows patients to report in near real-time their health status and specific COVID-19 symptoms to a nominated clinician team, and, using their findings, the researchers will co-produce with patients a targeted intervention for Long COVID that’s tailored to individual patient needs. 

Discover more about the development and validation of the symptom burden questionnaire for long COVID (SBQ-LC)

Our Credentials

21 CFR Part 11 compliant
Crest
Cyber Essentials Plus
ePrivacy App
ISO 13485
ISO 27001
Read More