Automatic emotion recognition in the voice of children with autism spectrum conditions

Recognition and analysis of affect from vocal signals has gained attention as the voice is our primary mean of communication with other people. Our voice implicitly carries information about ourselves, our current state and our attitude towards others in a dialogue. Individuals with autism spectrum conditions (ASC) have limited abilities in recognising and expressing emotions which might be an additional barrier for their social inclusion. Very few studies have investigated automatic emotion analysis of speech of children with ASC. This thesis advances the state of the art in the area by analysing to what extent acoustic features are relevant when children with ASC are expressing emotional states. Particular importance is given to the use of prosodic features which can be used in concrete applications, for instance, serious games where the child can interact and intuitively learn how to manipulate these parameter during a game. This analysis is extended to cover different languages and populations. Experimental results are presented on databases containing speech of children with ASC and typically developing children under the same conditions. Based on the acoustic analysis and feature relevance, this thesis also focuses on a real-life application of voice analysis and speech emotion recognition for assistive information communication technology (ICT) solutions aiming at the inclusion of children with ASC. In particular, an on-line voice analyser is developed and integrated into the perceptual serious game platform ASC-Inclusion. This platform is an effective educational intervention. Clinical evaluations showed an evident generalised improvement in socialisation and other symptoms present in those with an ASC. All in all this thesis is a first step towards the application of automatic speech emotion recognition in socially assistive technologies for children with ASC.