ML for brain-machine interfaces

We investigate various methods to control robotic assistive devices via brain interfaces.  We use surface EMG for controlling the grasp force of the hand as well as the position of the arm in a user-intuitive manner.  To this end, an adaptive system learns the correspondence between the human hand position and orientation and the muscular activity measured at the skin surface.  Thereby we can control to move the arm and grasp an object in teleoperation / telemanipulation.  Such control schemes are also applicable in rehabilitation and orthoses environments.

 

 

EMG and EEG

We use various machine-learning methodologies for processing peripheral neural data.  In particular, we focus on recurrent neural networks and variational autoencoders (VAE) to process and preprocess these data.  

We are furthermore investigating the control of such robots through human cortical implants. In this control scheme, neural signals recorded in the human motor cortex are decoded in continuous motion commands, that are executed by the robot. This combination of state-of-the-art robotics and advanced neuro-prosthesis allow a person with severe physical disabilities to physically interact with their environment again. Our results have been published in Nature in 2012. Further information can be found here.


Other interfaces, including invasive communication with the human peripheral nervous system as well as surface EEG control are within the realm of our research spectrum.

Picture of  Daniela Korhammer

Daniela Korhammer

alumni
Picture of  Jörn Vogel

Jörn Vogel

DLR: PhD candidate
BCI robot control
joern.vogeldlrde, +49 8153 28-2166
Picture of  Justin Bayer

Justin Bayer

alumni
bayersensedio
Picture of  Rachel Hornung

Rachel Hornung

DLR: PhD candidate
rehabilitation robotics
rachel.hornungdlrde
Picture of  Patrick van der Smagt

Patrick van der Smagt

current: Head of AI Research, data lab, VW Group

Previous: Director of BRML labs
fortiss, An-Institut der Technischen Universität München
Professor for Biomimetic Robotics and Machine Learning, TUM

Chairman of Assistenzrobotik e.V.
smagtbrmlorg



2016

    Holger Urbanek, Patrick van der Smagt (2016). iEMG: Imaging Electromyography. Journal of Electromyography and Kinesiology. 27 1--9.

2015

    J. Vogel, S. Haddadin, B. Jarosiewicz, J. D. Simeral, D. Bacher, M. M. Hochberg, J. P. Donoghue, P. van der Smagt (2015). An assistive decision-and-control architecture for force-sensitive hand--arm systems driven by human--machine interfaces. The International Journal of Robotics Research (IJRR).

2014

    Jörn Vogel, Sami Haddadin, John D Simeral, Sergey D Stavisky, Daniel Bacher, Leigh R Hochberg, John P Donoghue, Patrick van der Smagt (2014). Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. In Oussama Khatib and Vijay Kumar and Gaurav Sukhatme (Eds.) Experimental Robotics 79 125-136.

2013

    Claudio Castellini, Patrick van der Smagt (2013). Evidence of muscle synergies during human grasping. Biological Cybernetics. 107 (2), 233-245.
    Dominic Lakatos, Daniel Rüschen, Justin Bayer, Jörn Vogel, Patrick van der Smagt (2013). Identification of Human Limb Stiffness in 5 DoF and Estimation via EMG. In Desai, Jaydev P. and Dudek, Gregory and Khatib, Oussama and Kumar, Vijay (Eds.) Experimental Robotics 88 89-99.
    Joern Vogel, Justin Bayer, Patrick van der Smagt (2013). Continuous robot control using surface electromyography of atrophic muscles. Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems 845--850.

2012

    Leigh R. Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas Y. Masse, John D. Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Sydney S. Cash, Patrick van der Smagt, John P. Donoghue (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 485 372-377.

2011

    Vogel J, Castellini C, Smagt P van der (2011). EMG-Based Teleoperation and Manipulation with the DLR LWR-III. Proc. IROS---International Conference on Intelligent Robots and Systems 672-678.

2010

    Liu J, Simeral JD, Stavisky SD, Bacher D, Vogel J, Haddadin S, Smagt P van der, Hochberg LR, Donoghue JP (2010). Control of a robotic arm using intracortical motor signal by an individual with tetraplegia in the BrainGate2 trial. 40th Annual Meeting in Neuroscience (SFN2010)
    Vogel J, Haddadin S, Simeral J D, Stavisky S D, Bacher D, Hochberg L R, Donoghue J P, Smagt P van der (2010). Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. International Symposium on Experimental Robotics (ISER)

2009

    Castellini C, Smagt P van der (2009). Surface EMG in Advanced Hand Prosthetics. Biological Cybernetics. 100 (1), 35--47.

2008

    Castellini C, Smagt P van der, Sandini G, Hirzinger G (2008). Surface EMG for Force Control of Mechanical Hands. Proceedings - IEEE International Conference on Robotics and Automation 725--730.
    Maier S, Smagt P van der (2008). Surface EMG suffices to classify the motion of each finger independently. Proceedings of MOVIC. 9th International Conference on Motion and Vibration Control

2006

    Bitzer S, Smagt P van der (2006). Learning EMG control of a robotic hand: towards active prostheses. Proceedings 2006 IEEE International Conference on Robotics and Automation 2819-2823.