Why are robotic hands so clumsy?  In an attempt to solve this problem, we investigate the use of tactile sensors for robotic in-hand manipulation. In particular, based on a number of tactile sensors (the BioTac; the DLR tactile sensor; and the iCub tactile sensor), we want to create algorithms that can use such sensing to manipulate objects.  In cleartext: write, unscrew a bulb, or hand over a knife with a robotic hand.

During the task of making a cup of tea the constituent actions (e.g., sub-actions like “grasping the kettle”, “pouring water into the mug” etc.) should be recognized using methods of computer vision. Analysis bases on videos of subjects performing the task. AR should be on-line, intermediate solutions maybe off-line. Other signals (object forces and acceleration, data from Kinect©) are available and can eventually be used to support Action Recognition.

During the task of making a cup of tea, the performance of a set of subjects is monitored with various sensors: hand trajectories using motion capture, objects dynamics using acceleration and force sensors and gaze information using recordings of eye movements with scene video. In the master project algorithms for pattern recognition should to be developed. In particular, action errors that may happen during task execution should be detected in advance. Detection has to be demonstrates with off- line analysis, in future application error detection should be on-line.



In Machine Learning, we investigate methods to map high-dimensional non-linear data within a control process. Even though most of our data are related to the above fields of research, the methods we employ and develop are general methods, in which we combine deep belief networks with time sequence learning.

The use of surface electromyography (sEMG) for prosthetic control has been in place since the 1960's. We go a step further. On the one hand, we optimise the conditioning of the sEMG signal, and find new ways of relating it to limb movement. But we also look at different channels to control prosthetic and assistive robotic devices, including central nervous system implants.

In biomechanics, we create details models of human kinematic and dynamic properties of arms, hands, fingers, and legs. These models are needed to understand which properties of human movement are intrinsic---caused by muscles, tendons, ligaments and bones---and which are controlled by the nervous system. Our resulting models are used in the construction and control of novel robotic systems, including prosthetic hands and robotic arms and legs.

Limb rehabilitation and assistive robotics are paramount applications of the techniques developed in biomimetic robotics. We focus upon human-computer interfaces to aid the disabled regain the lost limb functionality. In our view, both rehabilitation and prosthetics rely on re-establishing the sensori-motor loop with the missing limb. This includes both ways: feed-forward control by detecting the user''s will to move and sensorial feedback by transducing digital readings to feelings.

Picture of  Axel von Arnim

Axel von Arnim

fortiss: software architect
Human Brain Project - Neurorobotics
Picture of  Moritz August

Moritz August

TUM: MSc candidate
deep autoencoder networks
Picture of  Adrià Puigdomènech Badia

Adrià Puigdomènech Badia

TUM: student
convolutional neural networks
Picture of  Justin Bayer

Justin Bayer

TUM: PhD candidate
time series learning
bayer.justingooglemailcom
Picture of  Nutan Chen

Nutan Chen

TUM: PhD candidate
hand modelling
Picture of  Rémy Degenne

Rémy Degenne

TUM: MSc candidate
cNN for one-class learning
Picture of  Frederik Diehl

Frederik Diehl

TUM: student assistant
tactile force learning
Picture of  Hannes Höppner

Hannes Höppner

DLR: PhD candidate
human impedance
hannes.hoeppnerdlrde, +49 8153 28-1062
Picture of  Rachel Hornung

Rachel Hornung

DLR: PhD candidate
rehabilitation robotics
rachel.hornungdlrde
Picture of  Haris Jabbar

Haris Jabbar

TUM: student
grip force modelling
Picture of  Sören Jentzsch

Sören Jentzsch

fortiss: PhD candidate
Human Brain Project: spiking networks
Picture of  Maximilian Karl

Maximilian Karl

TUM: MSc candidate
fast robust PCA
Picture of  Daniela Korhammer

Daniela Korhammer

TUM: PhD candidate
ML-based movement models
korhammdin.tumde
Picture of  Artur Lohrer

Artur Lohrer

TUM: research associate
tactile sensor fusion
Picture of  Marvin Ludersdorfer

Marvin Ludersdorfer

TUM: MSc candidate
mechatronics
Picture of  Saahil Ognawala

Saahil Ognawala

TUM: MSc candidate
recurrent neural networks
Picture of  Christian Osendorfer

Christian Osendorfer

TUM: PhD candidate
unsupervised learning, deep networks
osendorfin.tumde
Picture of  Thomas Rückstiess

Thomas Rückstiess

TUM: PhD candidate
reinforcement learning and design
rueckstiin.tumde
Picture of  Patrick van der Smagt

Patrick van der Smagt

Director of BRML labs
fortiss, An-Institut der Technischen Universität München
Professor for Biomimetic Robotics and Machine Learning, TUM
Chairman of Assistenzrobotik e.V.
smagtbrmlorg, +49 89 289-25793
Picture of  Benedikt Staffler

Benedikt Staffler

MPI/TUM: PhD candidate
cNN for connectome
Picture of  Georg Stillfried

Georg Stillfried

DLR: PhD candidate
kinematics of the human hand
georg.stillfrieddlrde
Picture of  Michael Strohmayr

Michael Strohmayr

DLR: postdoc
the DLR artificial skin
michael.strohmayrdlrde
Picture of  Sebastian Urban

Sebastian Urban

TUM: PhD candidate
learning skin data
surbantumde, +49 89 289-25794
Picture of  Holger Urbanek

Holger Urbanek

DLR: PhD candidate
EMG conditioning
holger.urbanekdlrde, +49 8153 28-2450
Picture of  Jörn Vogel

Jörn Vogel

DLR: PhD candidate
BCI robot control
joern.vogeldlrde, +49 8153 28-2166



Your name could be here

want to join our team? check out the positions on the left.



Below our 15 most recent publications. If you need more, follow the link. And note: All downloadable PDFs are for personal use only. Please do not redistribute.

Chen N, Urban S, Osendorfer C, Bayer J, Smagt P van der (2014). Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian Processes. Robotics and Automation (ICRA), 2014 IEEE International Conference on
Höppner H, Wiedmeyer W, Smagt P van der (2014). A new biarticular joint mechanism to extend stiffness ranges. Robotics and Automation (ICRA), 2014 IEEE International Conference on
Stillfried G, Hillenbrand U, Settles M, Smagt P van der (2014). MRI-based skeletal hand movement model. In Ravi Balasubramanian and Veronica J. Santos (Eds.) The Human Hand as an Inspiration for Robot Hand Development 95 49-75.
Vogel J, Haddadin S, Simeral J D, Stavisky S D, Bacher D, Hochberg L R, Donoghue J P, Smagt P van der (2014). Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. In Oussama Khatib and Vijay Kumar and Gaurav Sukhatme (Eds.) Experimental Robotics 79 125-136.
Höppner H, McIntyre J, Smagt P van der (2013). Task Dependency of Grip Stiffness---A Study of Human Grip Force and Grip Stiffness Dependency during Two Different Tasks with Same Grip Forces. PLOS ONE. 8 (12), e80889.
Franosch JP, Urban S, Hemmen JL van (2013). Supervised Spike-Timing-Dependent Plasticity: A Spatiotemporal Neuronal Learning Rule for Function Approximation and Decisions. Neural Computation. 25 (12), 3113-3130.
Fligge N, Urbanek H, Smagt P van der (2013). Relation between object properties and EMG during reaching to grasp. Journal of Electromyography and Kinesiology. 23 (2), 402-410.
Rückstieß T, Osendorfer C, Smagt P van der (2013). Minimizing Data Consumption with Sequential Online Feature Selection. International Journal of Machine Learning and Cybernetics. 4 (3), 235-243.
Braun DJ, Petit F, Huber F, Haddadin S, Smagt P van der, Albu-Schaffer A, Vijayakumar S (2013). Robots Driven by Compliant Actuators: Optimal Control under Actuation Constraints. IEEE Transactions on Robotics. 99 (5), 1--17.
Castellini C, Smagt P van der (2013). Evidence of muscle synergies during human grasping. Biological Cybernetics. 107 (2), 233-245.
Lakatos D, Rüschen D, Bayer J, Vogel J, Smagt P van der (2013). Identification of Human Limb Stiffness in 5 DoF and Estimation via EMG. In Desai, Jaydev P. and Dudek, Gregory and Khatib, Oussama and Kumar, Vijay (Eds.) Experimental Robotics 88 89-99.
Vogel J, Bayer J, Smagt P van der (2013). Continuous robot control using surface electromyography of atrophic muscles. Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems 845--850.
Urban S, Bayer J, Osendorfer C, Wesling G, Edin BB, Smagt P van der (2013). Computing grip force and torque from finger nail images using Gaussian processes. Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 4034--4039.
Bayer J, Osendorfer C, Urban S, Smagt P van der (2013). Training Neural Networks with Implicit Variance. Neural Information Processing 8227 132-139.
Osendorfer C, Bayer J, Smagt P van der (2013). Convolutional Neural Networks learn compact local image descriptors. Neural Information Processing 8228 624-630.