As Professor Socially Intelligent Computing at the University of Twente, my research interests cover both the machine analysis of human (conversational) behaviour and the generation of human-like (conversational) behaviour by virtual agents and robots. I am especially interested in the nonverbal and paraverbal aspects of dialogue and what these signals reveal about the mental state (cognitive, affective, social). These topics are explored both from a computational perspective and as basic research in the humanities, reflecting my training as a computational linguist. But my interests are much broader as you may learn from these pages and my publications.

After my studies of Linguistics, Computer Science and Computational Linguistics at the University of Antwerp I moved to the Institute of Dutch Lexicology in Leyden to develop tools for enriching natural language databases. After a couple of years I went on to the Utrecht University and got involved in the big European project Eurotra on Machine Translation. After coordinating a follow-up EU project on the automatic translation of collocations, I started my PhD project on a logical approach to natural language analysis and parsing (Type Logical Grammar).

In 1999, I started working at the University of Twente switching to research on embodied dialogue systems (aka virtual agents or embodied conversational agents). This made my interests shift from pure linguistic analysis to body language, from text analysis to real-time human-machine interaction, and from the logical analysis to a much broader concern with emotion and social relations in mediated human-human and in human-computer interaction.

Over the years, I have had the opportunity to expand investigations in the full bandwith of multimodal communication. In the BrainGain project, for instance, I was able to look at brain signals as a mode of human computer interaction and in the COMMIT/ project VIEWW I am exploring interaction through “touch”.

I have been involved in several European projects such as Humaine, AMI and AMIDA, COST 2102, Semaine, SSPNET, SERA, Aria Valuspa and De-ENIGMA. Dialogue management, human behaviour modeling, cognitive modeling, human-robot interaction, methodological issues in corpus studies and annotation, are some of the topics dealt with in these projects. An interesting Dutch national projects I was involved in include BrainGain in which neural correlates of a user’s experience are explored for Human Computer Interaction using Brain Computing Interfaces and GATE dealing with cognitive models of virtual agents in training applications.

I was the project leader of the COMMIT project Interaction for Universal Access in which we not only explored the use of virtual agents for training, but also investigated interactive playgrounds for children.

Topics of research: embodied conversational agents, social signal processing, ambient intelligence, affective computing spoken dialogue systems, human computer interaction, backchannels, brain computer interfaces, interactive playgrounds, virtual and augmented reality.