Reaching out to Autism through the Use of Novel Technology: the IM-TWIN project

According to the World Health Organization (WHO), the global incidence of Autism Spectrum Disorder (ASD) is estimated to be approximately 1 in 100 children worldwide. One way to support these children originated in the European project "IM-TWIN: from Intrinsic Motivations to Transitional Wearable INtelligent companions for autism spectrum disorder". The 3-year project, which ran from November 2020 to October 2023, involved five European partners from Italy, Portugal, France and the Netherlands. 

    As explained by the project coordinator Gianluca Baldassarre, from the Institute of Cognitive Sciences and Technologies (National Research Council of Italy, ISTC-CNR), “The project aimed to create a technological system, called ‘IM-TWIN’, to support neurodevelopmental therapists and neuropsychiatrists in the early treatment of Neurodevelopmental Disorders (NDD), with particular reference to ASD”. 

    The IM-TWIN system is formed of 3 components: innovative, interactive soft toys that look like animals, called ‘Transitional Wearable Companions (TWC)’, a wearable sensorised T-shirt for the detection of a child’s physiological parameters, and sensorised camera glasses, for the detection of eye contact between child and therapist.

The project aimed to create a technological system, called ‘IM-TWIN’, to support neurodevelopmental therapists and neuropsychiatrists. - Gianluca Baldassarre, Project Coordinator

     Beste Ozcan, researcher from ISTC-CNR and inventor of the TWCs, explains that “smart soft toys can be used to stimulate the curiosity and engagement of autistic children. For example, the TWC “Panda PlusMe” can produce gratifying sensory responses, such as coloured lights, amusing sounds and mild vibrations when its paws are caressed. Another example is the TWC “Octopus X-8” that is able to produce responses that are different when its tentacles are touched by the child versus the therapist. These features allow the therapist to set up play activities that train a child's social competencies: e.g.  imitation, eye contact, joint attention and turn-taking”.

    PLUX Wireless Biosignals, a Portuguese partner company, highlights that “the sensorised T-shirt was designed to collect physiological data in very young ASD children. In particular, it allows the detection of galvanic skin response, heart rate, skin temperature, and body motion -- all parameters related to the stress level of the subject”. In the pilot test, the T-shirt proved to be able to collect reliable data on children involved in therapeutic play activities. In this regard, the researchers headed by Egon L. van den Broek from Utrecht University in the Netherlands, developed an innovative algorithm, called “fast Continuous Wavelet Transformation - fCWT”. The fCWT outperforms currently existing algorithms in extracting meaningful patterns from intrinsically noisy physiological signals. After this processing, the data can potentially be used to train an Artificial Intelligence (AI) to ‘understand’ the affective states of autistic children during the therapeutic activities. These states can be challenging to comprehend in ASD.

         Finally, the sensorised camera glasses were developed to use AI and Computer Vision to reliably detect eye contact between therapist and child. This behaviour is crucial for monitoring social engagement in ASD children.
  All experimental activities involving ASD and neurotypical children were performed  respectively at the University of Rome Sapienza, Department of Human Neuroscience, Section of Child Neuropsychiatry, supervised by Vincenzo Guidetti and Carla Sogos, and at a team headed by Kevin O’Regan at the Learning Planet Institute, in Paris.

As remarked by Gianluca Baldassarre “the IM-TWIN system represents an innovative new technological tool built for ASD early intervention, that can help the therapist to monitor and stimulate children through play-like activities, thus facilitating the development of social skills”.

Supporting material

• Project website: https://im-twin.eu/ 
• brochure: project booklet
• promotional video: project technological outcomes

Project legal details

• Project No: 952095
• Timeframe: 1 November 2020 - 31 October 2023
• Programme: H2020-EU.1.2.2. - FET Proactive
• Topic: FETPROACT-EIC-06-2019 - EIC Transition to Innovation Activities
• Funding scheme: RIA - Research and Innovation Action
• Cordis fact sheet: https://cordis.europa.eu/project/id/952095