logo EDITE Sujets de doctorat

Les premières secondes comptent : gérer les premières impressions pour un agent virtuel plus engageant

Sujet proposé par
Directeur de thèse:
Doctorant: Beatrice BIANCARDI
Unité de recherche UMR 7222 Institut des Systèmes Intelligents et de Robotique

Domaine: Sciences et technologies de l'information et de la communication


This thesis is part of an ANR French-Switzerland project IMPRESSIONS that involves a French partner, CNRI-LTCI Telecom-ParisTech and a Swiss partner, Université de Genève. In any encounter the first moments are critical and the impressions that we form of others matter. These impressions tend to last and affect the interaction experience.

In human-human interaction first impressions can be formed by observing individual characteristics (or stereotypical traits), such as height, clothing and, more generally, visual appearance [Naumann et al. 2009; Argyle 1988; Miller et al. 2007]. However, first impressions can be formed also by observing someone behavior, such as facial expressions and body language (i.e. nonverbal behavior) [Riggio and Friedman 1986; Argyle 1988; Burgoon et al. 1984]. In the context of first encounters, individuals can carefully plan how to present themselves visually, but it may be difficult to have full control over all nonverbal cues during the interaction [DePaulo 1992].

Many researchers [Kervyn et al. 2013; Fiske et al., 2007] attempted to find which elements influence social perception, or in other words "the impression we form of others". Two variables or dimensions are most often proposed to describe this concept: warmth and competence [Fiske et al. 2007].

The goal of the project IMPRESSION is to build an anthropomorphic virtual character (ECA -Embodied Conversational Agent) able to make the best possible first impression on a user, thus effectively engaging him or her in an interaction. This goal will be realized by building an affective loop which ties the behavior of the ECA to the actual emotional reactions of the user facing it in real-time. This will give rise to an ECA capable of managing their first impressions on users. We focus on the identification and modeling of the nonverbal behavior, towards exhibiting, managing and maintaining impressions of two important socio¬cognitive dimensions in the first minutes of interaction with a user. These dimensions are warmth (i.e. being friendly, agreeable, engaging and approachable) and competence (i.e. appear skilled, knowledgeable on a given topic).


The thesis aims at designing, modeling and developing the virtual agent's expressed behavior in order to manage impressions along the warmth/competence dimensions on the user. The agent will be able to manage those impressions and attempt to maintain them during the interaction with the user by implementing an adaptation mechanism that takes advantage of the user's impressions assessment resulted from multimodal detection mechanisms provided by Universite de Geneve.

Several steps are foreseen:

• Creation of a Repertoire of Multimodal Behavior for Managing First Impression: define a repertoire of human multimodal behavior that can be modeled into the agent to manage the first impressions of the selected dimensions. This repertoire will be built on a literature review first, and then will take advantage of WOz studies results. The Woz studies are collected by Universite de Geneve.

• Creation of the Agent Mental Representation: create a module that can be referred as the 'Agent's Mind". First, it will be capable of storing the multimodal processed data provided by Universite de Geneve (impression assessment). These data represent the user's state as detected by the agent in terms of impressions that s/he has formed of it. Secondly, the agent's internal state, including its goals and beliefs on the user, and behavior plans are represented.

• Creation of the Agent's Impression Management Module: The agent's first impressions management will be controlled by this module. This task will develop in two steps aimed at creating a Function Planner and a Behavior Planner. The architecture of our agent is based on the existing SAIBA framework which is implemented by in the VIB system. In the first step, the intended communicative functions (i.e. intents of managing the selected impressions of warmth and competence) are produced by a Function Planner component. Secondly, a Behavior Planner will generate the agent's multimodal behavior supporting the intended functions and that will be exhibited by the virtual agent. The intent generation (i.e. communicative functions for managing impressions) will use the information retrieved from the agent's mind about the internal state of the agent (e.g. goals and beliefs on user) and the adaptive feedback provided in input and describing the user's affective state (provided by Universite de Geneve). The resulting communicative intents (e.g. greet the user, ask a question, etc...) will be supported by multimodal behavior generated in the second step by the behavior planner selecting those behaviors from the repertoire of multimodal behaviors.

• Evaluation of the First Impressions management: first we aim at evaluating the quality of our conversational agent in terms of communicative skills and believability of exhibited behavior. Secondly we need to validate the effectiveness of the agent's impression management capabilities. The effectiveness of the agent will be evaluated by measuring the duration of the interaction (i.e. the agent was effective to maintain user's attention) and we will evaluate whether users form the impressions that the agent attempt to manage on them using questionnaires. Thirdly, we will evaluate possible impacts that managing first impressions might have on users. Finally we will verify that users prefer an agent with impressions management skills over an agent which does not manage users' impressions.