Research Projects

Ibotics AVATAR, XPRIZE Utwente 400K; Avatar Xprize team from University of Twente, TNO, Halodi, ETH Zurich. 2019-2021, video

Kids conversing with a media robot, SIDN and NL Institute for Sound and Vision 2019-2021 500K

Principal Investigator (PI).

Kids conversing with a media robot investigates how we can design child-robot interaction in a responsible manner: not only informative in a way that children understand and like, but also with attention to transparency, privacy and awareness about the use of artificial intelligence

DE-ENIGMA EU H2020 (RIA), No 688835, Socially Assistive Robot for Children with ASD, 2016-2020, 4M€/600K€

Coordinator / Head PI

The project has built a robot that interacts with children with autism and helps them learn about showing emotions. Autism affects children in many ways, particularly their communication and interaction skills: they can have difficulties recognising and understanding people’s behaviours and expressing their own emotions. Conventionally, foundational skills like emotion recognition are taught face-to-face with an adult. The EU-funded project DE-ENIGMA developed a less socially demanding and more entertaining option that uses artificial intelligence.

TERESA EU FP7 STREP, No 611153, Telepresence Reinforcement-learning Social Agent, 2013-2016, (4M€ total/ 700K€)

Principle Investigator (PI)

The TERESA project aimed to develop a telepresence robot of unprecedented social intelligence, thereby helping to pave the way for the deployment of robots in settings such as homes, schools, and hospitals that require substantial human interaction. In telepresence systems, a human controller remotely interacts with people by guiding a remotely located robot, allowing the controller to be more physically present than with standard teleconferencing. We are developing a new telepresence system that frees the controller from low-level decisions regarding navigation and body pose in social settings. Instead, TERESA will have the social intelligence to perform these functions automatically. In particular, TERESA will semi-autonomously navigate among groups, maintain face-to-face contact during conversations, and display appropriate body-pose behaviour.

SQUIRREL EU FP7 IP, No 610532, Clearing clutter bit by bit FP7 Integrated Project, 2013-2017, (8.5M€ total/ 900K€)

Principal Investigator (PI)

Clutter in an open world is a challenge for many aspects of robotic systems, especially for autonomous robots deployed in unstructured domestic settings, affecting navigation, manipulation, vision, human robot interaction and planning. SQUIRREL addresses these issues by actively controlling clutter and incrementally learning to extend the robot’s capabilities while doing so. SQUIRREL is inspired by a user driven scenario, that exhibits all the rich complexity required to convincingly drive research, but allows tractable solutions with high potential for exploitation. We will advance science w.r.t. manipulation, where we will incrementally learn grasp affordances with a dexterous hand; segmenting and learning objects and object category models from a cluttered scene; localisation and navigation in a crowded and changing scene based on incrementally built 3D environment models; iterative task planning in an open world; and engaging with multiple users in a dynamic collaborative task.

EASEL, EU FP7-ICT-2013-10, No. 611971, Expressive Agents for Symbiotic Education and Learning FP7 STREP, 2013-2016, (4M€ total/640K€)

(Principle Investigator, PI)

EASEL will explore and develop a theoretical understanding of human-robot symbiotic interaction (HRSI) where symbiosis is defined as the capacity of the robot and the person to mutually influence each other, and alter each other’s behaviour over different time-scales (for instance, within encounters and across encounters). Symbiosis requires that the robot can read, and be responsive to, the behaviour and emotional state of the person, and adapt its own behaviour to take into account the user social context and extract knowledge from long sequences of behavioural interaction and changing its action responses accordingly.

COMMIT P3, Dutch National Project: SENSEI,  motivational coach for amateur sports, 2013-1017,  (2.5M€ total/ 400K€)

(Principle Investigator)

We aim to help adhere and motivate the society towards a healthy tomorrow by empowering the individuals with a personalized health and fitness coach which relies on the power of the smart phones (sensors and computation). Given to the increasing percentage of unhealthy population in today’s society which can be attributed to sedentary unhealthy lifestyle and excessive food intake, preventive and recuperative healthcare becomes utmost important. Thus, a coaching application, which gives real-time, personalized coaching considering ones physical, mental and emotional state will go a long way. For this project the challenge is how effectively we can motivate people towards a healthy and fit regime by giving them personalized feedback taking into account their physical, mental and social wellbeing.

SPENCER: FP7-ICT-9-2.1, Nr. 600877, Challenge 2: Cognitive Systems and Robotics FP7-Integrated Project, 2013-2016, (4.3M total/505K€)

(Principle Investigator, PI)

Cognitive Systems and Robotics SPENCER will break new ground for cognitive systems in populated environments. While there is an increasing focus on making robots more socially aware, related approaches are still limited in their capacity to perceive, model and learn human social behavior and respond with appropriate actions in real-time from mobile platforms. SPENCER will systematically address these problems and advance the fields of perception of individual humans and groups, normative human behavior learning and modeling, socially-aware mapping, and socially-aware task, motion and interaction planning under real-world conditions.

FROG: FP7-ICT-2011-7, Challenge 2: Cognitive Systems and Robotics, FP7 STREP nr. 288235, 2011-2014, (2.5M total/ 500K€)

(Coordinator, Head PI)

Cognitive Systems and Robotics FROG proposes to develop a guide robot with a winning personality and behaviors that will engage tourists in a fun exploration of outdoor attractions. The work encompasses innovation in the areas of vision-based detection, robotics design and navigation, human-robot interaction, affective computing, intelligent agent architecture and dependable autonomous outdoor robot operation. The FROG robots’ fun personality and social visitor-guide behaviors aim to enhance the user experience and increase knowledge transfer as information is offered through multi-sensory interaction. FROG adapts its behavior to the users through vision-based detection of human engagement and interest. FROG plans to build a highly dependable robot that operates autonomously in populated outdoor real-world settings.

ACCOMPANY: ICT-2011.5.4 ICT, No 287624, Acceptable Robotics Companions for Ageing and Wellbeing FP7 STREP, 2011-2014, (3.5M€ total/ 360K€)

Principal Investigator (PI)

ICT for Ageing and Wellbeing. ACCOMPANY provides a robotic companion as part of an intelligent environment. It helps older people to live independently at home by providing physical, cognitive and social assistance in everyday home tasks. ACCOMPANY contributes to the re-enablement of the user by assisting him or her to carry out tasks independently. Services to the user will be delivered through socially interactive, acceptable and empathic interaction, building on computational models of social cognition and interaction. With Care-O-bot® 3, a state of the art service robot platform, user requirements and user acceptance of the robot will be assessed. User study results will be fed back to make the technology better suit user demands and preferences. Three test sites in three different European countries (UK, the Netherlands and France) as well as a dedicated showcase will ensure an extensive evaluation process which also takes into account cultural differences. ACCOMPANY will specify and benchmark design and ethical guidelines for service robots for the elderly.

Contributed to Grant Writing:

DHRS-CIM: Distributed Human-Robot System for Chemical Incident Management.

(Researcher)

The overall aim of this project is knowledge exchange between industrial and academic organizations in the field of Disaster Management; working towards implementing / prototyping, an intelligent decision support system for humans to manage chemical incidents. This system can be described as a Distributed Human-Robot System for Chemical Incident Management (DHRS-CIM). 250k€.

SmartINSIDE Eureka/Eurostars project, Smartphones with Innovative Navigation Service for an unique InDoor Experience , 2010-2011, European Community, 126 k€.

DIADEM Distributed information acquisition and decision-making for environmental management (EU FP7 2008-2011) (450k)

(Co-PI)

The Prime Objective of the Diadem project is to create an ICT system for collaborative decision making that effectively supports the protection of the population and the environment against chemical hazards in industrial areas.

MOCATOUR (2009-2012), Amsterdam Living Labs A Pieken in de Delta project. The  (Mobile Cultural access for tourists) project seeks to establish novel computational methods to facilitate tourists with contextualized and experience-based access to information while they freely explore a city. (450k)

ICIS Interactive Collaborative Information Systems (3024 BSIK project 2004-2009)

The aim of ICIS is to develop better techniques for making complex information systems more intelligent and supportive in decision making situations. The ICIS project combines information technology research and development with research in the human sciences and integrates the resulting knowledge and insights. Although the research results are generally applicable, ICIS focuses on two key applications domains: traffic management and crisis response.

CYBURG (2002, 2003). Digitale kenniswijk Cyburg, financed by the city council of Amsterdam and EU funds. In this project we evaluated cultural differences in requirements for e-government services in the multi-cultural city district Zeeburg.