We focus on machine learning, mostly deep learning and recurrent neural networks, and its application to movement representation, robotics, and sensor integration.

We use and further develop deep learning, deep recurrent neural networks, convolutional neural networks, autoencoders, etc. and use them in various data related to movement, robotics, and sensory processing.

Using deep learning, we do end-to-end learning on visual and tactile data.  We also develop methods for anomaly detection.

Using our machine-learning methodologies, we model movement in latent domains and use those in movement prediction, analysis, and control.  These models are combined with biomechanical and motor control models.

Limb rehabilitation and assistive robotics are paramount applications of the techniques developed in biomimetic robotics. Using our machine-learning methods, we focus upon human-computer interfaces to aid the disabled regain the lost limb functionality. In our view, both rehabilitation and prosthetics rely on re-establishing the sensori-motor loop with the missing limb.

We'd like you to devise and consolidate a system which robustly predicts robot and human arm movement based on visual data.  You will be working with variational autoencoders, recurrent neural networks, and convolutional neural networks in a lab which focuses on machine learning, deep learning, and robotics.

You are asked to work on real robotic data to understand control parameters of a light-weight robotic arm. 

Why are robotic hands so clumsy?  In an attempt to solve this problem, we investigate the use of tactile sensors for robotic in-hand manipulation. In particular, based on a number of tactile sensors (the BioTac; the DLR tactile sensor; and the iCub tactile sensor), we want to create algorithms that can use such sensing to manipulate objects. In cleartext: write, unscrew a bulb, or hand over a knife with a robotic hand.

Picture of  Adam Kosiorek

Adam Kosiorek

TUM: MSc candidate
visual movement coding
Picture of  Axel von Arnim

Axel von Arnim

fortiss: software architect
Human Brain Project - Neurorobotics
Picture of  Baris Kalayibay

Baris Kalayibay

TUM: BSc candidate
CNN segmentation of medical data
Picture of  Benedikt Staffler

Benedikt Staffler

MPI/TUM: PhD candidate
cNN for connectome
Picture of  Christian Osendorfer

Christian Osendorfer

Picture of  Daniela Korhammer

Daniela Korhammer

Picture of  Grady Jensen

Grady Jensen

LMU: Ph.D. candidate
movement modelling
Picture of  Hannes Höppner

Hannes Höppner

DLR: postdoc
human impedance
hannes.hoeppnerdlrde, +49 8153 28-1062
Picture of  Holger Urbanek

Holger Urbanek

DLR: postdoc
imaging EMG
holger.urbanekdlrde, +49 8153 28-2450
Picture of  Jörn Vogel

Jörn Vogel

DLR: PhD candidate
BCI robot control
joern.vogeldlrde, +49 8153 28-2166
Picture of  Justin Bayer

Justin Bayer

Picture of  Markus Kühne

Markus Kühne

TUM: PhD candidate
MR-compatible haptic interfaces
Picture of  Marvin Ludersdorfer

Marvin Ludersdorfer

fortiss: PhD candidate
anomaly detection
Picture of  Maximilian Karl

Maximilian Karl

TUM: PhD candidate
efficient inference
Picture of  Maximilian Soelch

Maximilian Soelch

TUM: PhD candidate
robot control with deep learning
Picture of  Nutan Chen

Nutan Chen

TUM: PhD candidate
hand modelling
Picture of  Patrick van der Smagt

Patrick van der Smagt

current: Head of AI Research, data lab, VW Group

Previous: Director of BRML labs
fortiss, An-Institut der Technischen Universität München
Professor for Biomimetic Robotics and Machine Learning, TUM

Chairman of Assistenzrobotik e.V.
Picture of  Rachel Hornung

Rachel Hornung

DLR: PhD candidate
rehabilitation robotics
Picture of  Sebastian Urban

Sebastian Urban

TUM: PhD candidate
learning skin data
surbantumde, +49 89 289-25794
Picture of  Wiebke Koepp

Wiebke Koepp

TUM: PhD candidate
novel activation functions in neural networks

Your name could be here

want to join our team? check out the positions on the left.

Below our 15 most recent publications. If you need more, follow the link. And note: All downloadable PDFs are for personal use only. Please do not redistribute.

Baris Kayalibay, Grady Jensen, Patrick van der Smagt (2017). CNN-based Segmentation of Medical Imaging Data. arXiv.
Holger Urbanek, Patrick van der Smagt (2016). iEMG: Imaging Electromyography. Journal of Electromyography and Kinesiology. 27 1--9.
Agneta Gustus, Patrick van der Smagt (2016). Evaluation of joint type modelling in the human hand. Journal of Biomechanics. 49 (13), 3097–3100.
Christopher Wolf, Maximilian Karl, Patrick van der Smagt (2016). Variational Inference with Hamiltonian Monte Carlo. arXiv.
Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt (2016). Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data. arxiv.
The Theano Development Team (2016). Theano: A Python framework for fast computation of mathematical expressions. arxiv.
Nutan Chen, Maximilian Karl, Patrick van der Smagt (2016). Dynamic Movement Primitives in Latent Space of Time-Dependent Variational Autoencoders. Proc. 16th IEEE-RAS International Conference on Humanoid Robots
Herke van Hoof, Nutan Chen, Maximilian Karl, Patrick van der Smagt, Jan Peters (2016). Stable Reinforcement Learning with Autoencoders for Tactile and Visual Data. Proc. IEEE International Conference on Intelligent Robots and Systems (IROS)
Wiebke Koepp, Patrick van der Smagt, Sebastian Urban (2016). A Differentiable Transition Between Additive and Multiplicative Neurons. International Conference on Learning Representations (ICLR)
Maximilian Sölch, Justin Bayer, Marvin Ludersdorfer, Patrick van der Smagt (2016). Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series. International Conference on Learning Representations (ICLR)
J. Vogel, S. Haddadin, B. Jarosiewicz, J. D. Simeral, D. Bacher, M. M. Hochberg, J. P. Donoghue, P. van der Smagt (2015). An assistive decision-and-control architecture for force-sensitive hand--arm systems driven by human--machine interfaces. The International Journal of Robotics Research (IJRR).
Sebastian Urban, Marvin Ludersdorfer, Patrick van der Smagt (2015). Sensor Calibration and Hysteresis Compensation with Heteroscedastic Gaussian Processes. IEEE Sensors. 15 (11), 6498--6506.
Maximilian Karl, Justin Bayer, Patrick van der Smagt (2015). Efficient Empowerment.
Nutan Chen, Justin Bayer, Sebastian Urban, Patrick van der Smagt (2015). Efficient movement representation by embedding Dynamic Movement Primitives in Deep Autoencoders. Proc. 2015 IEEE-RAS International Conference on Humanoid Robots
Justin Bayer, Maximilian Karl, Daniela Korhammer, Patrick van der Smagt (2015). Fast Adaptive Weight Noise. arXiv