Emotional Robotics

Ravi Vaidyanathan/Suresh Devasahayam

It is now well-accepted that for a robot to interact effectively with humans, it should manifest some form of believable behaviour establishing appropriate social expectations and regulating interaction.  We are working with the Bristol Robotics Laboraotory (BRL) to introduce a simplified robotic system capable of emotial (affective) human-robot communication without the complexity of full facial actuation, and to empirically assess human response to that robot for social interaction.  In addressing these two issues, we provide a basis for the larger goal of developing a mechanically simple platform for deeper human-robot cooperation as well as a method to quantify, neurologically, human response to affective robots.  The scope encompasses: 1) the development of hybrid-face humanoid robot capable of emotive response, including pupil dilation, and 2) the testing of hybrid-face through mapping human response to the robot qualitatively (interview/feedback) and quantitatively (EEG ERPs) based on human-robot testing.  Principal contributions lie in modelling of emotion affect space and mapping neurological response to robot emotion.  


Bristol-Elumotion Robotic Torso (BERT2) social robot with an expressive face meant to help researchers design intelligent systems capable of ensuring mutual trust, safety, and effective cooperation with human beings

Brain Activity Map;  The BERT2 robot made Neutral, Angry, Stern and Happy facial expressions at a human subject test group with concurrent brain activitry (EEG) recorded.  The color map shows the parts of the brain that were most active when the person viewed the robot emotion.  The spatial EEG topography of evoked negative N170 response to robot facial expressions demonstrates similar brain activity to that of recognizing the same human facial expressions


The Biomechatronics Laboratory is collaboraiton with Emotix on the companion social robot Miko to commercialize this work.  Miko has been released to the public in India, and will be availalbe soon internationally.  See http://www.emotix.in/#2 for more information on this exciting product

Miko: Meet Miko, India's first companion robot, who will soon be avaialbe internationally


Focus area: Psychosocial Engagement with Mental Health in Developing Nations

Cognitive impairment caused by mental illness is a world-wide epidemic.  In 2015, dementia costs alone were estimated at $818 billion worldwide; this figure will grow to $2 trillion annually by 2030, which could easily overwhelm medical and social care systems as they stand today [WHO 2017].   It is particularly a problem in low resource settings in middle and low income countries (LICs, MICs) where medical attention is sparse. Maximizing psychosocial interventions is vital in mental health for patient support and to arrest degeneration. Such interventions may take the form of cognitive stimulation, physical activity, and/or art-mediated therapies.  While its impact on well-being, cognition and daily functioning is well acknowledged, its frequency is not always practical, traceable, or controllable for dementia patients.  Recent research has introduced the use of interactive ‘social’ robots to engage and stimulate dementia patients through verbal, semantic, and/or physical interaction. Social robots enable creation of diverse behaviours and customization, enable intervention addressing inter-individual differences (a well-known success factor in dementia care), and provide a means of documenting patient response through recording of robot actions.

Despite this potential, clinical utility of robotic tools for actual intervention such as cognitive stimulation remains in its infancy; furthermore it is nearly non-existent in MICs and LICs.  We propose to extend our team’s work in social robots [Emotix, 2018; Vaidyanathan 2018; Bazo 2011], quantification of neural stimulation to social robot engagement [Craig 2011], robots in rehabilitative therapy [Burridge 2017; Woodward 2017], and brain-robot interfaces [Angeles 2017; Mamun 2015; Mace 2014] to initiate the development of robotic cognitive engagement tools for patients to improve mental health in both India and the UK.  We will lay a foundation for the development of the first inexpensive clinically viable robotic engagement system that will be assessed with dementia patients and medical professionals in-clinic in India.


Professor Suresh Devasahayam, Christian Medical College Vellore, Vellore, India

Professor Chris Melhuish, Dr Appolinaire Etoundi, Bristol Robotics Laboraotry, Bristol, UK

Professor Christopher James, University of Warwick, Coventry, UK

Emotix, Mumbai India


UK Dementia Research Institute (DRI)

UK Grand Challenge Research Fund (GCRF)

European Union Framework 7 Pesearch Program


Angeles P, Tai Y, Pavese N, Vaidyanathan R (2017) “Assessing Parkinson’s disease motor symptoms using supervised learning algorithms” [abstract], Movement Disorders, 32 (suppl 2)

Bazo D, R Vaidyanathan R., A Lenz, C Melhuish, “Design and Testing of Hybrid Expressive Face for the BERT2 Humanoid Robot”, (2011) Proceedings of IEEE International Conference on Intelligent Robots and Systems (IROS), pp 5317-5322

Broekens, J., Heerink, M., and Rosendal, H. (2009). Assistive social robots in elderly care: a review. Gerontechnology 8, 94–103

Burridge J, A Lee, R Turk, M Stokes, J Whitall, R Vaidyanathan, P Clatworthy, A M Hughes, C Meagher, E Franco, L Yardley, “Tele-health, wearable sensors and the Internet: Will they improve stroke outcomes through increased intensity of therapy, motivation and adherence to rehabilitation programs?” Journal of Neurologic Physical Therapy, 1-25, 2017

Cidav Z, Marcus S C. , Mandell S C. , “Implications of Childhood Autism for Parental Employment and Earnings”, Pediatrics, 129, 4, 2012

Craig R, R Vaidyanathan, CJ James, C Meluish, (2011) “Assessment of Human Response to Robot Facial Expressions through Visually Evoked Potentials”, Proceedings of the IEEE International Conference on Humanoid Robots, 647-652

Dias A, Patel V. Closing the treatment gap for dementia in India. Indian J Psychiatry 2009; 51 Suppl 1:93–97

Emotix Cooperation, MIKO: The Companion Robot, (http://www.emotix.in/)

Inoue T., et al. (2012) “Field-based development of an information support robot for persons with dementia”, Technology and Disability, 20, 263-271

Khan F, Dementia in India, AP.J of Psychol Medicine, 2011, 12(2)

Knapp M., Autism costs. Health and Social Care, 2012

Mace M., Abdullah-al-Mamun, K., Naeem A.A., Gupta, L., Wang, S., Vaidyanathan, R., (2014) "A heterogeneous framework for real-time decoding of bioacoustic signals: Applications to assistive interfaces and prosthesis control", Expert Systems with Applications, 40, 13, 5049-5060

Mamun K, M Mace, M Lutman, J Stein, X Liu, T Aziz, R Vaidyanathan, S Wang, (2015) “Movement decoding using neural synchronisation and inter-hemispheric connectivity from deep brain local field potentials”, Journal of Neural Engineering, 12, 5, 1-18, 2015

Martín, F., Agüero, C. E., Cañas, J. M., Valenti, M., and Martínez-Martín, P. (2013). Robotherapy with dementia patients. Int. J. Adv. Robotic Syst. 10, 1–7.

Miura T., et al., (2016) “Need and impressions of communication robots for seniors with slight physical and cognitive disabilities: Evaluation using system usability scale”, Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp 4088-4092

Morgan P, et al. (2017), “An Emerging Framework to Inform Effective Design of Human-Machine Interfaces for Older Adults Using Connected Autonomous Vehicles”, Proceedings of the International Conference on Applied Human Factors and Ergonomics, 325-334

Mordoch, E., Osterreicher, A., Guse, L., Roger, K., and Thompson, G. (2013). Use of social commitment robots in the care of elderly people with dementia: a literature review. Maturitas 74, 14–20.

Moyle, W et al. (2017) “Use of a Robotic Seal as a Therapeutic Tool to Improve Dementia Symptoms: A Cluster-Randomized Controlled Trial.” Journal of the American Medical Directors Association 18 9 (2017): 766-773

Petersen, S., Houston, S., Qin, H., Tague, C., and Studley, J. (2017). The utilization of robotic pets in dementia care. J. Alzheimers Dis. 55, 569-57

Prince M, Guerchet M, Prina M. (2015). The Epidemiology and Impact of Dementia - Current State and Future Trends. WHO Thematic Briefing. Available from: http://www.who.int/mental_health/neurology/dementia/dementia_thematicbrief_epidemiology.pdf

Prince M, Livingston G, Katona C. Mental health care for elderly in Low Income countries: A Health systems approach. World Psychiatry 2007; 6:5-13

Raghuraman S, Lakshminarayanan M,Vaitheswaran S, Rangaswamy T, (2017) Cognitive Stimulation Therapy for Dementia: Pilot Studies of Acceptability and Feasibility of Cultural Adaptation for India, Am J Geriatr Psychiatry 25:9

Rouaix N, Retru-Chavastel L, Rigaud A-S, Monnet C, Lenoir H and Pino M (2017) Affective and Engagement Issues in the Conception and Assessment of a Robot-Assisted Psychomotor Therapy for Persons with Dementia. Front. Psychol. 8:950, 1-15

Shaji KS, Jotheeswaran AT, Girish N, Bharath S, Dias A, Pattabiraman M. The Dementia India Report: Prevalence, Impact, Costs and Services for Dementia. NewDelhi: Alzheimer’s & Related Disorders Society of India 2010; 44–52

UK AsR (https://www.dementiastatistics.org/statistics/hospitals/), accessed Sept 21 2018

Valentí Soler,M., Agüera-Ortiz, L., Olazarán Rodríguez, J.,Mendoza Rebolledo, C., Pérez Muñoz, A., Rodríguez Pérez, I., et al. (2015). Social robots in advanced dementia. Front. Aging Neurosci. 7:133

Vaidyanathan R, Vanaswami S, Iyengar P, (2018) System for person detection and identification using multiple sensors on a social robot, submitted, Indian Patent Office, service action identification pending

Wada, K., and Shibata, T. (2007). Living with seal robots: its sociopsychological and physiological influences on the elderly at a care house. IEEE Trans. Robot 23, 972–980

Woodward R, S Shefelbine, R Vaidyanathan, (2018) “Gait analysis using pervasive motion tracking and mechanomyography Fusion”, IEEE Transactions on Mechatronics, 22,5, pp 2022-2033, 2017

World Health Organization (2017), Global action plan on the public health response to dementia 2017-2025