Picture of  Patrick van der Smagt

Patrick van der Smagt

Director of BRML labs
fortiss, An-Institut der Technischen Universität München
Professor for Biomimetic Robotics and Machine Learning, TUM
Chairman of Assistenzrobotik e.V.
addressGermany
emailsmagtbrmlorg

In heading a research lab focussing on machine learning and its application in robotics, biomimetics and sensory data processing, my goal is to develop the techniques to model and use (human) movement.

The slides of my Keynote on end-to-end learning at the 2015 IROS conference are available here.

Visit my blog.

awards

Best Paper Award, Int Conf on Neural Information Processing (ICONIP 2014)
King-Sun Fu Best 2013 Transactions on Robotics Paper Award (2014)
Harvard Medical School/MGH Martin Research Prize (2013)
Erwin Schrödinger Award, Helmholtz Gesellschaft (2012)
SfN BCI Award Finalist (2012)
TUM Leonardo da Vinci Award (2008)
IEEE Best Paper Awards
Beckmann Institute Fellowship (1995)
NACEE Fellowship

in the press

various sources, e.g. NY Times, May 16, 2012: on a brain-controlled robotics experiment
Bayerische Rundfunk, May, 2012: "Wenn Rechner immer intelligenter werden", Radio Wissen
n-tv, June 10, 2010: EMG-controlled robotics
pinc, May 18, 2009, "biorobotics"
Het Financieele Dagblad, May, 2009
Discovery Channel, April 2009: "Future Homes"
3Sat, April 22, 2007: "Z wie Zukunft"
RTLII, March, 2007: "Welt der Wunder"
Pro7, Nov. 5, 2006: "Wunderwelt Wissen"
Abendzeitung, Oct. 28, 2006: Bestnoten für Forscher und Unternehmen
ZDF: Heute Journal, Sep. 15, 2006: Interview
ORF: "Newton" Science report, April 30, 2006: report on advanced prostheses
Süddeutsche Zeitung, Mar. 03, 2006: "Künstliche Hand am Computer entwickelt"
Süddeutsche Zeitung, Jan. 26, 2006: "Direkter Draht zum Hirn"

We were and are funded by various sources, including:
DFG project "SPP autonomous learning" (2012-2015)
NEUROBOTICS (EC project, 2005-2009)
NINAPRO (Swiss project, 2010-2013)
SENSOPAC (EC project, 2006-2010)
STIFF (EC project, 2009-2011)
THE (EC project, 2010-2014)
VIACTORS (EC project, 2009-2012)

on publishing

In June 2012 I resigned as editor of Neural Networks (Elsevier). Having worked with Neural Networks for almost 20 years, I have come to realise that the publication model propagated by behind-paywall publishers no longer combines with my own views of publication DOs and DONTs.  In particular, I have decided to move away from classical publication methods towards open access publishing, now that such alternatives are maturing.

After so many years of research and publishing, it is clear that only a peer-to-peer (double-open) review system with open access to the publications can be fair and unbiased.

I am currently still editor of Biological Cybernetics, as the open access model is being supported by Springer. However, also that large publishing house will have to rethink their approach to scientific publication before long.

Reviewing is good.  But open publication is an alternative.  My blog is an attempt to solve, for my own benefit, this publication issue.




Holger Urbanek, Patrick van der Smagt (2016). iEMG: Imaging Electromyography. Journal of Electromyography and Kinesiology. 27 1--9.
Agneta Gustus, Patrick van der Smagt (2016). Evaluation of joint type modelling in the human hand. Journal of Biomechanics.
Maximilian Karl, Maximilian Soelch, Justin Bayer, Patrick van der Smagt (2016). Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data. arxiv.
Herke van Hoof, Nutan Chen, Maximilian Karl, Patrick van der Smagt, Jan Peters (2016). Stable Reinforcement Learning with Autoencoders for Tactile and Visual Data. Proc. IEEE International Conference on Intelligent Robots and Systems (IROS)
Wiebke Koepp, Patrick van der Smagt, Sebastian Urban (2016). A Differentiable Transition Between Additive and Multiplicative Neurons. International Conference on Learning Representations (ICLR)
Maximilian Sölch, Justin Bayer, Marvin Ludersdorfer, Patrick van der Smagt (2016). Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series. International Conference on Learning Representations (ICLR)
J. Vogel, S. Haddadin, B. Jarosiewicz, J. D. Simeral, D. Bacher, M. M. Hochberg, J. P. Donoghue, P. van der Smagt (2015). An assistive decision-and-control architecture for force-sensitive hand--arm systems driven by human--machine interfaces. The International Journal of Robotics Research (IJRR).
Sebastian Urban, Marvin Ludersdorfer, Patrick van der Smagt (2015). Sensor Calibration and Hysteresis Compensation with Heteroscedastic Gaussian Processes. IEEE Sensors. 15 (11), 6498--6506.
Maximilian Karl, Justin Bayer, Patrick van der Smagt (2015). Efficient Empowerment.
Nutan Chen, Justin Bayer, Sebastian Urban, Patrick van der Smagt (2015). Efficient movement representation by embedding Dynamic Movement Primitives in Deep Autoencoders. Proc. 2015 IEEE-RAS International Conference on Humanoid Robots
Justin Bayer, Maximilian Karl, Daniela Korhammer, Patrick van der Smagt (2015). Fast Adaptive Weight Noise. arXiv
Zoltán A. Milacskí, Marvin Ludersdorfer, Patrick van der Smagt, Andräs Lörincz (2015). Robust Detection of Anomalies via Sparse Methods. Proc. ICONIP
Nutan Chen, Sebastian Urban, Justin Bayer, Patrick van der Smagt (2015). Measuring Fingertip Forces from Camera Images for Random Finger Poses. Proc. IEEE International Conference on Robotics and System (IROS)
Hannes Höppner, Markus Grebenstein, Patrick van der Smagt (2015). Two-dimensional orthoglide mechanism for revealing areflexive human arm mechanical properties. Proc. IEEE International Conference on Intelligent Robots and Systems (IROS 2015)
Sebastian Urban, Patrick van der Smagt (2015). A Neural Transfer Function for a Smooth and Differentiable Transition Between Additive and Multiplicative Interactions. arXiv
P. Fischer, A. Dosovitskiy, E Ilg, P Haeusser, C Hazirbas, V. Golkov, P. van der Smagt, D. Cremers, T. Brox (2015). FlowNet: Learning Optical Flow with Convolutional Networks. Proc. ICCV
Hornung R, Urbanek H, Klodmann J, Osendorfer C, Smagt P van der (2014). Model-free robot anomaly detection. Proc. IEEE/RSJ International Conference on Robotics and Systems (IROS)
Justin Bayer, Christian Osendorfer, Daniela Korhammer, Nutan Chen, Sebastian Urban, Patrick van der Smagt (2014). On Fast Dropout and its Applicability to Recurrent Networks. Proceedings of the International Conference on Learning Representations
Christian Osendorfer, Hubert Soyer, Patrick van der Smagt (2014). Image Super-Resolution with Fast Approximate Convolutional Sparse Coding. Neural Information Processing 8836
Nutan Chen, Sebastian Urban, Christian Osendorfer, Justin Bayer, Patrick van der Smagt (2014). Estimating finger grip force from an image of the hand using Convolutional Neural Networks and Gaussian Processes. Robotics and Automation (ICRA), 2014 IEEE International Conference on
Hannes Höppner, Wolfgang Wiedmeyer, Patrick van der Smagt (2014). A new biarticular joint mechanism to extend stiffness ranges. Robotics and Automation (ICRA), 2014 IEEE International Conference on
Georg Stillfried, Ulrich Hillenbrand, Marcus Settles, Patrick van der Smagt (2014). MRI-based skeletal hand movement model. In Ravi Balasubramanian and Veronica J. Santos (Eds.) The Human Hand as an Inspiration for Robot Hand Development 95 49-75.
Jörn Vogel, Sami Haddadin, John D Simeral, Sergey D Stavisky, Daniel Bacher, Leigh R Hochberg, John P Donoghue, Patrick van der Smagt (2014). Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. In Oussama Khatib and Vijay Kumar and Gaurav Sukhatme (Eds.) Experimental Robotics 79 125-136.
Hannes Höppner, Joseph McIntyre, Patrick van der Smagt (2013). Task Dependency of Grip Stiffness---A Study of Human Grip Force and Grip Stiffness Dependency during Two Different Tasks with Same Grip Forces. PLOS ONE. 8 (12), e80889.
N. Fligge, H. Urbanek, P. van der Smagt (2013). Relation between object properties and EMG during reaching to grasp. Journal of Electromyography and Kinesiology. 23 (2), 402-410.
Rückstieß T, Osendorfer C, Smagt P van der (2013). Minimizing Data Consumption with Sequential Online Feature Selection. International Journal of Machine Learning and Cybernetics. 4 (3), 235-243.
David J. Braun, Florian Petit, Felix Huber, Sami Haddadin, Patrick van der Smagt, Alin Albu-Schaffer, Sethu Vijayakumar (2013). Robots Driven by Compliant Actuators: Optimal Control under Actuation Constraints. IEEE Transactions on Robotics. 99 (5), 1--17.
Claudio Castellini, Patrick van der Smagt (2013). Evidence of muscle synergies during human grasping. Biological Cybernetics. 107 (2), 233-245.
Dominic Lakatos, Daniel Rüschen, Justin Bayer, Jörn Vogel, Patrick van der Smagt (2013). Identification of Human Limb Stiffness in 5 DoF and Estimation via EMG. In Desai, Jaydev P. and Dudek, Gregory and Khatib, Oussama and Kumar, Vijay (Eds.) Experimental Robotics 88 89-99.
Joern Vogel, Justin Bayer, Patrick van der Smagt (2013). Continuous robot control using surface electromyography of atrophic muscles. Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems 845--850.
Sebastian Urban, Justin Bayer, Christian Osendorfer, Göran Wesling, Benoni B. Edin, Patrick van der Smagt (2013). Computing grip force and torque from finger nail images using Gaussian processes. Proc. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 4034--4039.
Justin Bayer, Christian Osendorfer, Sebastian Urban, Patrick van der Smagt (2013). Training Neural Networks with Implicit Variance. Neural Information Processing 8227 132-139.
Christian Osendorfer, Justin Bayer, Sebastian Urban, Patrick van der Smagt (2013). Convolutional Neural Networks learn compact local image descriptors. Neural Information Processing 8228 624-630.
Leigh R. Hochberg, Daniel Bacher, Beata Jarosiewicz, Nicolas Y. Masse, John D. Simeral, Joern Vogel, Sami Haddadin, Jie Liu, Sydney S. Cash, Patrick van der Smagt, John P. Donoghue (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. 485 372-377.
Agneta Gustus, Georg Stillfried, Judith Visser, Henrik Jorntell, Patrick van der Smagt (2012). Human hand modelling: kinematics, dynamics, applications. Biological Cybernetics. 106 (11-12), 741-755.
D. J. Braun, F. Petit, S. Haddadin, P. van der Smagt, A. Albu-Schäffer, S. Vijayakumar (2012). Optimal Torque and Stiffness Control in Compliantly Actuated Robots. Proc. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems 2801--2808.
Atzori M, Gijsberts A, Heynen S, Mittaz-Hager A, Deriaz O, Smagt P van der, Castellini C, Caputo B, Müller H (2012). Building the NINAPRO Database: A Resource for the Biorobotics Community. IEEE International Conference on Biomedical Robotics and Biomechatronics 1258--1265.
Francesca Cordella, Francesco Di Corato, Loredana Zollo, Bruno Siciliano, Patrick van der Smagt (2012). Patient performace evaluation using kinect and Monte Carlo-based finger tracking. IEEE International Conference on Biomedical Robotics and Biomechatronics 1967--1972.
Dominikus Gierlach, Agneta Gustus, Patrick van der Smagt (2012). Generating marker stars for 6D optical tracking. IEEE International Conference on Biomedical Robotics and Biomechatronics 147--152.
Nadine Fligge, Joe McIntyre, Patrick van der Smagt (2012). Minimum jerk for human catching movements in 3D. Proc. IEEE International Conference on Biomedical Robotics and Biomechatronics 581--586.
Bayer J, Osendorfer C, Smagt P van der (2012). Learning sequence neighbourhood metrics. In Villa, Alessandro and Duch, Włodzisław and \'Erdi, Péter and Masulli, Francesco and Palm, Günther (Eds.) Artificial Neural Networks and Machine Learning – ICANN 2012 7552 531-538.
Rückstiess T, Osendorfer C, Smagt P van der (2011). Sequential feature selection for classification. In Dianhui Wang and Mark Reynolds (Eds.) AI 2011: Advances in Artificial Intelligence 7106
Osendorfer C, Schlüter J, Schmidhuber J, Smagt P van der (2011). Unsupervised learning of low-level audio features for music similarity estimation. Workshop on Learning Architectures, Representations, and Optimization for Speech and Visual Information Processing, ICML 2011
Smagt P van der (2011). Neue Entwicklungen in der Rehabilitation von Handfunktionsstörungen: Humanrobotik. In Dennis A. Novak (Eds.) Handfunktionsstörungen in der Neurologie: Klinik und Rehabilitation 433-451.
Vogel J, Castellini C, Smagt P van der (2011). EMG-Based Teleoperation and Manipulation with the DLR LWR-III. Proc. IROS---International Conference on Intelligent Robots and Systems 672-678.
Lakatos D, Petit F, Smagt P van der (2011). Conditioning vs. Excitation Time for Estimating Impedance Parameters of the Human Arm. Proceedings of the 11th IEEE-RAS International Conference on Humanoid Robots 636-642.
Höppner H, Lakatos D, Urbanek H, Castellini C, Smagt P van der (2011). The Grasp Perturbator: Calibrating human grasp stiffness during a graded force task. Proc. ICRA---International Conference on Robotics and Automation 3312-3316 .
Castellini C, Smagt P van der (2011). Preliminary evidence of dynamic muscular synergies in human grasping. Proceedings of ICAR - International Conference on Advanced Robotics 28-33.
Liu J, Simeral JD, Stavisky SD, Bacher D, Vogel J, Haddadin S, Smagt P van der, Hochberg LR, Donoghue JP (2010). Control of a robotic arm using intracortical motor signal by an individual with tetraplegia in the BrainGate2 trial. 40th Annual Meeting in Neuroscience (SFN2010)
Vogel J, Haddadin S, Simeral J D, Stavisky S D, Bacher D, Hochberg L R, Donoghue J P, Smagt P van der (2010). Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. International Symposium on Experimental Robotics (ISER)
Strohmayr M, Saal H, Potdar A, Smagt P van der (2010). The DLR Touch Sensor I: A Flexible Tactile Sensor for Robotic Hands based on a Crossed-Wire Approach. Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems 897-903.
Höppner H, Lakatos D, Urbanek H, Smagt P van der (2010). The Arm-Perturbator: Design of a Wearable Perturbation Device to measure Limb Impedance. International Conference on Applied Bionics and Biomechanics (ICABB)
Stillfried G, Smagt P van der (2010). Movement model of a human hand based on magnetic resonance imaging (MRI). International Conference on Applied Bionics and Biomechanics (ICABB)
Gruijl J de, Smagt P van der, Zeeuw C de (2009). Anticipatory grip force control using a cerebellar model. Neuroscience. 162 (3), 777-786.
Smagt P van der, Grebenstein M, Urbanek H, Fligge N, Strohmayr M, Stillfried G, Parrish J, Gustus A (2009). Robotics of human movements. Journal of physiology, Paris. 103 (3-5), 119-132.
Castellini C, Smagt P van der (2009). Surface EMG in Advanced Hand Prosthetics. Biological Cybernetics. 100 (1), 35--47.
Grebenstein M, Smagt P van der (2008). Antagonism for a highly anthropomorphic hand-arm system. Advanced Robotics. 22 (1), 39-55.
Smagt P van der, Stillfried G (2008). Using MRT data to compute a hand kinematic model. Proc. 9th International Conference on Motion and Vibration Control (MOVIC)
Panzer H, Eiberger O, Grebenstein M, Wolf S, Schaefer P, Smagt P van der (2008). Human motion range data optimizes anthropomorphic robotic hand-arm system design. Proc. 9th International Conference on Motion and Vibration Control (MOVIC)
Castellini C, Smagt P van der, Sandini G, Hirzinger G (2008). Surface EMG for Force Control of Mechanical Hands. Proceedings - IEEE International Conference on Robotics and Automation 725--730.
Arbib M, Metta G, Smagt P van der (2008). Neurorobotics: From Vision to Action. In B. Siciliano and O. Khatib (Eds.) Springer Handbook of Robotics 1453--1480.
Maier S, Smagt P van der (2008). Surface EMG suffices to classify the motion of each finger independently. Proceedings of MOVIC. 9th International Conference on Motion and Vibration Control
Smagt P van der, Hirzinger G (2006). NEUROBOTICS: The fusion of neuroscience and robotics. Proceedings ISR-2006, Joint Conference on Robotics / ROBOTIK
Bitzer S, Smagt P van der (2006). Learning EMG control of a robotic hand: towards active prostheses. Proceedings 2006 IEEE International Conference on Robotics and Automation 2819-2823.
Urbanek H, Albu-Schäffer A, Smagt P van der (2004). Learning from demonstration: Repetitive movements for autonomous service robotics. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 3495-3500.
Smagt P van der, Bullock D (2002). Guest editorial for Special Issue on Scalable Applications of Neural Networks to Robotics. Applied Intelligence. 17 7--10.
Peters J, Smagt P van der (2002). Searching a Scalable Approach to Cerebellar Based Control. Applied Intelligence. 17 (1), 11-33.
Smagt P van der (2000). Benchmarking Cerebellar Control. Robotics and Autonomous Systems. 32 237--251.
Smagt P van der, Hirzinger G (2000). The cerebellum as computed torque model. Fourth International Conference on knowledge-Based Intelligent Engineering Systems & Applied Technologies IEEE: 760--763.
Smagt P van der (1998). Cerebellar Control of Robot Arms. Connection Science. 10 301--320.
Smagt P van der, Hirzinger G (1998). Solving the Ill-Conditioning in Neural Network Learning. In J. Orr and K. Müller (Eds.) Neural Networks: Tricks of the Trade 193--206.
Smagt P van der, Hirzinger G (1998). Why Feed-Forward Networks are in a Bad Shape. In L. Niklasson and M. Bodén and T. Ziemke (Eds.) Proceedings of the 8th International Conference on Artificial Neural Networks Springer: 159--164.
Fischer M, Smagt P van der, Hirzinger G (1998). Learning Techniques in a Dataglove Based Telemanipulation System for the DLR Hand. Transactions of the IEEE International Conference on Robotics and Automation 1603--1608.
Omidvar O, Smagt P van der (1997). Neural Systems for Robotics. Academic Press: Boston, Massachusetts
Smagt P van der, Groen F (1997). Visual Feedback in Motion. In O. Omidvar and P. van der Smagt (Eds.) Neural Systems for Robotics 37--73.
Smagt P van der, Bullock D (1997). Can Artificial Cerebellar Models Compete to Control Robots?.
Smagt P van der (1997). Teaching a robot to see how it moves. In Antony Browne (Eds.) Neural Network Perspectives on Cognition and Adaptive Robotics 195--219.
Smagt P van der, Groen F, Schulten K (1996). Analysis and control of a rubbertuator arm. Biological Cybernetics. 75 (5), 433--440.
Smagt P van der (1996). A robot arm is neurally controlled using monocular feedback. Self Learning Robots, IEE Colloquium on IEEE: 9/1--9/3.
Smagt P van der, Kröse B (1995). Using Many-Particle Decomposition to get a Parallel Self-Organising Map. In J. van Vliet (Eds.) Proceedings of the 1995 Conference on Computer Science in the Netherlands 241-249.
Smagt P van der, Groen F, Kröse B (1995). A Monocular Robot Arm can be Neurally Positioned. In U. Rembold and R. Dillmann and L. O. Hertzberger and T. Kanade (Eds.) Proceedings of the 1995 International Conference on Intelligent Autonomous Systems IOS Press: 123--130.
Smagt P van der, Groen F (1995). Approximation with neural networks: Between local and global approximation. Proceedings of the 1995 International Conference on Neural Networks II:1060-II:1064.
Smagt P van der, Dev A, Groen F (1995). A visually guided robot and a neural network join to grasp slanted objects. In B. Kappen and S. Gielen (Eds.) Proceedings of the 3rd SNN Symposium on Neural Networks Foundation for Neural Networks, Nijmegen: Nijmegen, Netherlands 144-150.
Smagt P van der (1995). Visual Robot Arm Guidance using Neural Networks. Ph.D. thesis: Dept of Computer Systems, University of Amsterdam
Jansen A, Smagt P van der, Groen F (1995). Nested Networks for Robot Control. In A. F. Murray (Eds.) Applications of Neural Networks 221--239.
Smagt P van der (1994). Simderella: a robot simulator for neuro-controller design. Neurocomputing. 6 (2), 281--285.
Smagt P van der (1994). Minimisation methods for training feed-forward networks. Neural Networks. 7 (1), 1--11.
Hesselroth T, Sarkar K, Smagt P van der, Schulten K (1994). Neural network control of a pneumatic robot arm. IEEE Transactions on Systems, Man, and Cybernetics. 24 (1), 28--38.
Smagt P van der, Groen F, Groenewoud F van het (1994). The locally linear nested network for robot manipulation. Proceedings of the IEEE International Conference on Neural Networks 2787--2792.
Smagt P van der, Groen F, Groenewoud F van het (1994). Robotic Hand-Eye Coordination using Multi-Resolution Linear Perceptron Representation. In H. M. Groenboom and H. W. Klijn Hesselink and M. M. Lankhorst (Eds.) Proceedings of the 1994 Groningen Student Conference on Computer Science 85--92.
Kröse B, Smagt P van der (1994). An Introduction to Neural Networks. University of Amsterdam: Amsterdam, The Netherlands
Smagt P van der, Schulten K (1993). Control of pneumatic robot arm dynamics by a neural network. World congress on neural networks III/180-III/183.
Smagt P van der, Groen F, Kröse B (1993). Robot hand-eye coordination using neural networks.
Kröse B, Smagt P van der, Groen F (1993). A one-eyed self-learning robot manipulator. In G. Bekey and K. Goldberg (Eds.) Neural networks in robotics 19-28.
Groen F, Kröse B, Smagt P van der, Bartholomeus MGP, Noest AJ (1993). Neural Networks for robot eye-hand coordination. In S. Gielen and B. Kappen (Eds.) Artificial neural networks 211-218.
Smagt P van der, Kröse B, Groen F (1992). A self-learning controller for monocular grasping. Proceedings of the 1992 IEEE/RSJ International Conference on Intelligent Robots and Systems 177-182.
Smagt P van der, Kröse B, Groen F (1992). A Cyclops Learns to Grasp. Proceedings of the Second Symposium on Neural Networks 88.
Smagt P van der, Jansen A, Groen F (1992). Interpolative robot control with the nested network approach. IEEE Int. Symposium on Intelligent Control 475-480.
Jansen A, Smagt P van der, Groen F (1992). High-precision robot control: The nested network. In I. Aleksander and J. Taylor (Eds.) Artificial Neural Networks 2 583-586.
Smagt P van der, Kröse B (1991). A Real-Time Learning Neural Robot Controller. In T. Kohonen and K. Mäkisara and O. Simula and J. Kangas (Eds.) Proceedings of the 1991 International Conference on Artificial Neural Networks 351--356.
Groen F, Kröse B, Smagt P van der (1991). Parallel Distributed Processing in Autonomous Robot Systems. Proceedings of the 1991 Symposium on Neural Networks 24--25.
Smagt P van der (1990). A Comparative Study of Neural Network Algorithms Applied to Optical Character Recognition. Proceedings of the Third International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems 1037--1044.