Muscle Sounds in Gesture Recognition
Samuel Wilson
Muscle Sounds in Gesture Recognition for Prosthetic Control and Robot Teleoperation
This project focuses on creating a multifunctional control system based on Mechanomyography (MMG). MMG is the term given to the microvibrations which are generated during a muscle contraction. By placing microphones on the surface of the skin, these vibrations can be recorded. In this project we use the vibrations generated by the muscles in the forearm to determine what gestures the hand is performing.
Project Overview
Each sensor suite contains an IMU, which provides world referenced quaternions. These are used both to filter the MMG results, and to directly control other devices such as computers and robots.
Computer/Robot Control
The IMUs and MMGs provide both analogue and digital channels of data, meaning the the controller can be used from complex tasks.
IR Outputs
Infrared functionality was added to the suite to allow it to operate as a universal remote. Here it controls a TV and a remote control candle.
Prosthetic Control
We are lucky that Alex Lewis is able to visit to test our designs and give feedback. If you aren't familiar with his story, or if you are, please visit his website >Here<
Relevant Publications
- Upper-limb prosthetic control using wearable multichannel mechanomyography
- Gesture Recognition Through Classification of Acoustic Muscle Sensing for Prosthetic Control
- Hand gesture recognition with convolutional neural networks for the multimodal UAV control
- Other
Sponsors
US Office of Naval Research Global (ONR-G)
UK Centre for Blast Injury Studies
Enginering and Physical Sciences Research Council (EPSRC) Centre for Doctoral Training (CDT) in Neurotechnology
UK-India Educational Research Initiative (UKIERI), British Council
Undergraduate Work
My previous work on gesture recognition has been using smart gloves. Here an IMU on each finger and an IMU on the back of the hand were used to work out the movements of the hand in real time.