The SSPNet-Mobile Corpus: from the detection of non-verbal cues to the inference of social behaviour during mobile phone conversations

[摘要] Mobile phones are one of the main channels of communication in contemporary society.However, the effect of the mobile phone on both the process of and, also, the non-verbalbehaviours used during conversations mediated by this technology, remain poorly understood.This thesis aims to investigate the role of the phone on the negotiation process as well as,the automatic analysis of non-verbal behavioural cues during conversations using mobiletelephones, by following the Social Signal Processing approach. The work in this thesisincludes the collection of a corpus of 60 mobile phone conversations involving 120 subjects, development of methods for the detection of non-verbal behavioural events (laughter,fillers, speech and silence) and the inference of characteristics influencing social interactions(personality traits and conflict handling style) from speech and movements while using themobile telephone, as well as the analysis of several factors that influence the outcome ofdecision-making processes while using mobile phones (gender, age, personality, conflicthandling style and caller versus receiver role).The findings show that it is possible to recognise behavioural events at levels well abovechance level, by employing statistical language models, and that personality traits and conflicthandling styles can be partially recognised. Among the factors analysed, participant role(caller versus receiver) was the most important in determining the outcome of negotiationprocesses in the case of disagreement between parties. Finally, the corpus collected for theexperiments (the SSPNet-Mobile Corpus) has been used in an international benchmarkingcampaign and constitutes a valuable resource for future research in Social Signal Processingand more generally in the area of human-human communication.

[发布机构] University:University of Glasgow;Department:School of Computing Science

[关键词] Artificial intelligence, machine learning, non-verbal behavioural cues, social behaviour, personality traits, conflict handling style, negotiation, smartphones