We focus on machine learning, mostly deep learning and recurrent neural networks, and its application to movement representation, robotics, and sensor integration.

We use and further develop deep learning, deep recurrent neural networks, convolutional neural networks, autoencoders, etc. and use them in various data related to movement, robotics, and sensory processing.

Using deep learning, we do end-to-end learning on visual and tactile data.  We also develop methods for anomaly detection.

Using our machine-learning methodologies, we model movement in latent domains and use those in movement prediction, analysis, and control.  These models are combined with biomechanical and motor control models.

Limb rehabilitation and assistive robotics are paramount applications of the techniques developed in biomimetic robotics. Using our machine-learning methods, we focus upon human-computer interfaces to aid the disabled regain the lost limb functionality. In our view, both rehabilitation and prosthetics rely on re-establishing the sensori-motor loop with the missing limb.



We'd like you to devise and consolidate a system which robustly predicts robot and human arm movement based on visual data.  You will be working with variational autoencoders, recurrent neural networks, and convolutional neural networks in a lab which focuses on machine learning, deep learning, and robotics.

You are asked to work on real robotic data to understand control parameters of a light-weight robotic arm. 

Why are robotic hands so clumsy?  In an attempt to solve this problem, we investigate the use of tactile sensors for robotic in-hand manipulation. In particular, based on a number of tactile sensors (the BioTac; the DLR tactile sensor; and the iCub tactile sensor), we want to create algorithms that can use such sensing to manipulate objects. In cleartext: write, unscrew a bulb, or hand over a knife with a robotic hand.

Patrick moved and took his lab.  Our new webspace is at argmax.ai.




Your name could be here

want to join our team? check out the positions on the left.



Below our 15 most recent publications. If you need more, follow the link. And note: All downloadable PDFs are for personal use only. Please do not redistribute.

Benedikt Staffler, Manuel Berning, Kevin M Boergens, Anjali Gour, Patrick van der Smagt, Moritz Helmstaedter (2017). SynEM, automated synapse detection for connectomics. eLife. 2017;6:e26414
Baris Kayalibay, Grady Jensen, Patrick van der Smagt (2017). CNN-based Segmentation of Medical Imaging Data. arXiv.
Rui Zhao, Ali Haider, Patrick van der Smagt (2017). Two-Stream RNN/CNN for Action Recognition in 3D Videos. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2017)
Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt (2017). Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data. International Conference on Learning Representations (ICLR)
Holger Urbanek, Patrick van der Smagt (2016). iEMG: Imaging Electromyography. Journal of Electromyography and Kinesiology. 27 1--9.
Agneta Gustus, Patrick van der Smagt (2016). Evaluation of joint type modelling in the human hand. Journal of Biomechanics. 49 (13), 3097–3100.
Christopher Wolf, Maximilian Karl, Patrick van der Smagt (2016). Variational Inference with Hamiltonian Monte Carlo. arXiv.
Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt (2016). Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data. arxiv.
The Theano Development Team (2016). Theano: A Python framework for fast computation of mathematical expressions. arxiv.
Nutan Chen, Maximilian Karl, Patrick van der Smagt (2016). Dynamic Movement Primitives in Latent Space of Time-Dependent Variational Autoencoders. Proc. 16th IEEE-RAS International Conference on Humanoid Robots
Herke van Hoof, Nutan Chen, Maximilian Karl, Patrick van der Smagt, Jan Peters (2016). Stable Reinforcement Learning with Autoencoders for Tactile and Visual Data. Proc. IEEE International Conference on Intelligent Robots and Systems (IROS)
Wiebke Koepp, Patrick van der Smagt, Sebastian Urban (2016). A Differentiable Transition Between Additive and Multiplicative Neurons. International Conference on Learning Representations (ICLR)
Maximilian Sölch, Justin Bayer, Marvin Ludersdorfer, Patrick van der Smagt (2016). Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series. International Conference on Learning Representations (ICLR)
J. Vogel, S. Haddadin, B. Jarosiewicz, J. D. Simeral, D. Bacher, M. M. Hochberg, J. P. Donoghue, P. van der Smagt (2015). An assistive decision-and-control architecture for force-sensitive hand--arm systems driven by human--machine interfaces. The International Journal of Robotics Research (IJRR).
Sebastian Urban, Marvin Ludersdorfer, Patrick van der Smagt (2015). Sensor Calibration and Hysteresis Compensation with Heteroscedastic Gaussian Processes. IEEE Sensors. 15 (11), 6498--6506.