Georgia is appointed Assistant Professor at TU Darmstadt
Regularized Deep Signed Distance Fields for Reactive Motion Generation
Authors: Puze Liu, Kuo Zhang, Davide Tateo, Snehal Jauhri, Jan Peters and Georgia Chalvatzaki
Abstract
Autonomous robots should operate in real-world dynamic environments and collaborate with humans in tight spaces. A key component for allowing robots to leave structured lab and manufacturing settings is their ability to evaluate online and real-time collisions with the world around them. Distance-based constraints are fundamental for enabling robots to plan their actions and act safely, protecting both humans and their hardware. However, different applications require different distance resolutions, leading to various heuristic approaches for measuring distance fields w.r.t. obstacles, which are computationally expensive and hinder their application in dynamic obstacle avoidance use-cases. We propose Regularized Deep Signed Distance Fields (ReDSDF), a single neural implicit function that can compute smooth distance fields at any scale, with fine-grained resolution over high-dimensional manifolds and articulated bodies like humans, thanks to our effective data generation and a simple inductive bias during training. We demonstrate the effectiveness of our approach in representative simulated tasks for Whole-body control and safe Human-Robot Interaction (HRI) in shared workspaces. Finally, we provide proof of concept of a real-world application in a HRI handover task with a mobile manipulator robot.
Video demonstrations
Robot Learning of Mobile Manipulation with Reachability Behavior Priors
Authors: Snehal Jauhri, Jan Peters and Georgia Chalvatzaki
Abstract
Mobile Manipulation (MM) systems are ideal candidates for taking up the role of a personal assistant in unstructured real-world environments. Among other challenges, MM requires effective coordination of the robot’s embodiments for executing tasks that require both mobility and manipulation. Reinforcement Learning (RL) holds the promise of endowing robots with adaptive behaviors, but most methods require prohibitively large amounts of data for learning a useful control policy. In this work, we study the integration of robotic reachability priors in RL methods for accelerating the learning of MM. Namely, we consider the problem of optimal base placement and the subsequent decision of whether to activate the arm for reaching a 6D target. We derive Boosted Hybrid RL (BHyRL), a novel actor-critic algorithm that benefits from modeling Q-functions as a sum of residual approximators. Every time a new task needs to be learned, we can transfer our learned residuals and learn the component of the Q-function that is task-specific, hence, maintaining the task structure from prior behaviors.
Our Contributions
- We propose to use a hybrid action-space Reinforcement Learning algorithm for effectively tackling the need for discrete and continuous action decisions in Mobile Manipulation
- We learn a reachability behavioral prior for Mobile Manipulation that can speed up the learning process, and incentivize the agent to select kinematically reachable base poses when dealing with 6D reaching and fetching tasks
- We propose a new algorithm: Boosted Hybrid RL (BHyRL) for transferring knowledge from behavior priors by modelling Q-functions as sums of residuals, while also regularizing the policy learning in a trust-region fashion
Boosted Hybrid RL
- The concept of ‘boosting’ [1] is to combine many weak learners to create a single strong learner.
- To learn challenging base placement tasks, we first learn simpler reachability tasks and use the learnt behavior as a prior for accelerating the learning of subsequent tasks.
- To do this, the Q-function of every task is modelled as the sum of residuals learned on previous tasks [2],[3].
- Thus, we can progressively learn more difficult tasks while retaining the information and structure provided by the prior Q values.
- Additionally, we regularize the new task policy using a KL-divergence penalty with the previous policy.
Simulated tasks for 6D Reaching & Fetching
The agent learns progressively more challenging tasks and combines each of the learned behaviors:
6D_Reach_1m task
The agent needs to reach a 6D target in its vicinity (1-metre radius) by choosing an optimal base location and activating its arm for reaching
6D_Reach_5m task
The agent needs to navigate towards a 6D target that is up to 5 metres away. The 6D_Reach_1m behavior is used as a prior.
6D_Reach_3_obst task
Similar to the task above but now in the presence of 3 obstacles. The 6D_Reach_1m and 6D_Reach_5m behaviors are used as priors.
6D_Fetch_table/wall task
The agent needs to fetch an object placed on a table in the presence of a wall behind the table. The 6D_Reach_1m and 6D_Reach_5m behaviors are used as priors.
6D_Fetch_2_furniture task
The agent needs to fetch an object placed on a table in the presence of another furniture obstacle. The 6D_Reach_1m, 6D_Reach_5m and 6D_Reach_3_obst behaviors are used as priors.
6D_Fetch_multiobj task
The agent needs to fetch an object placed on a table without colliding with multiple other objects on the table. The 6D_Reach_1m and 6D_Reach_5m behaviors are used as priors.
Demonstration of zero-shot transfer of BHyRL policy
“Will this method work for my robot?”
Our training method can also work for other mobile manipulators such as the above ‘Fetch’ robot. Our codebase will be released at github.com/iROSA-lab/rlmmbp along with guidelines on how to train your own robot.
Acknowledgement
This research received funding from the German Research Foundation (DFG) Emmy Noether Programme (#448644653) and the RoboTrust project of the Centre Responsible Digitality Hessen, Germany.
References
- Yoav Freund, “Boosting a weak learning algorithm by majority”, Information and Computation, 121 (2):256–285, 1995.
- Samuele Tosatto, Matteo Pirotta, Carlo D’Eramo, and Marcello Restelli, “Boosted fitted q-iteration”, International Conference on Machine Learning, 2017.
- P. Klink, C. D’Eramo, J. Peters, and J. Pajarinen, “Boosted curriculum reinforcement learning,” in ICLR, 2022.
Making robots useful parts of our society
Professorship for Dr. Georgia Chalvatzaki
Since February, Georgia Chalvatzaki has been assistant professor for “Intelligent Robotic Systems for Assistance” at the Department of Computer Science. Chalvatzaki has been leading the iROSA research group since 2021 as part of the Emmy Noether Programme of the German Research Foundation. Previously, the 33-year-old researcher was a postdoctoral researcher in the Department of Intelligent Autonomous Systems in the Department of Computer Science. We asked Professor Chalvatzaki about her work:
Continue at: https://www.informatik.tu-darmstadt.de/fb20/aktuelles_fb20/fb20_news/news_fb20_details_251072.en.jsp
Co-Chair in IEEE WIE-RAS
Georgia will be serving as a co-chair of the IEEE RAS Women in Engineering group, along with Chair Karinne Ramirez Amaro and co-Chair Daniel Leidner.
The Women in Engineering – RAS (WIE-RAS) group was formed by the Member Activities Board (MAB) to inspire, engage and advance women in Robotics and Automation. The WIE committee organizes a series of periodic events and activities to promote the visibility of women leaders and to inspire young girls who want to get involved in engineering.
See more details: https://www.ieee-ras.org/women-in-engineering
Interview with PAL robotics
PAL robotics has interviewed Georgia about her career path, working with TIAGo, and her vision for iROSA.
Find the first part of the interview at: https://blog.pal-robotics.com/georgia-chalvatzaki-interview-part-1-starting-in-robotics-tiago-robot-home-robots-and-more/
Master Thesis: Discovering neural parts in objects with invertible NNs for robot grasping
In this thesis, we will investigate the use of 3D primitive representations in objects using Invertible Neural Networks (INNs). Through INNs we can learn the implicit surface function of the objects and their mesh. Apart from extracting the object’s shape, we can parse the object into semantically interpretable parts. In our work our main focus will be to segment the parts in objects that are semantically related to object affordances. Moreover, the implicit representation of the primitive can allow us to compute directly the grasp configuration of the object, allowing grasp planning.
Interested students are expected to have experience with Computer Vision and Deep Learning, but also know how to program in Python using DL libraries like PyTorch. The thesis will be co-supervised by Despoina Paschalidou (Ph.D. candidate at the Max Planck Institute for Intelligent Systems and the Max Planck ETH Center for Learning Systems). Highly motivated students can apply by sending an e-mail expressing your interest to georgia.chalvatzaki@tu-darmstadt.de, attaching your CV and transcripts.
References:
[1] Paschalidou, Despoina, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. “Neural Parts: Learning expressive 3D shape abstractions with invertible neural networks.” arXiv preprint arXiv:2103.10429 (2021).
[2] Karunratanakul, Korrawe, Jinlong Yang, Yan Zhang, Michael Black, Krikamol Muandet, and Siyu Tang. “Grasping Field: Learning Implicit Representations for Human Grasps.” arXiv preprint arXiv:2008.04451 (2020).
[3] Chao, Yu-Wei, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang et al. “DexYCB: A Benchmark for Capturing Hand Grasping of Objects.” arXiv preprint arXiv:2104.04631 (2021).
[4] Do, Thanh-Toan, Anh Nguyen, and Ian Reid. “Affordancenet: An end-to-end deep learning approach for object affordance detection.” In 2018 IEEE international conference on robotics and automation (ICRA), pp. 5882-5889. IEEE, 2018.
Talk @ UofT on accelerating Robot Skill Learning
My talk on “Accelerating Robot Skill Learning with Demonstrations and Models”, that I gave a few days ago at the AI in Robotics Seminar at the University of Toronto is now available online.
In this talk I go through our recent works “Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills” and “Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning” that will appear at ICRA2021.
hessian.AI Connectom fund
iROSA research group has received the hessian.AI Connectom fund, which promotes interdisciplinary research in the hessian.AI ecosystem.
This grant will allow researches from iROSA to work along researches from the Ubiquitous Knowledge Processing lab, led Prof. Dr. Gurevych.
Our project will investigate synergies between Robot Learning and Natural Language Processing towards Learning Long-Horizon Tasks by bridging Object-centric Representations to Knowledge Graphs.
AI Newcomer 2021
AI Newcomer 2021 of the category technical and engineering science is Dr. Georgia Chalvatzaki!
“AI advances robotic research for developing intelligent agents that interact safely and assist humans” – Georgia Chalvatzaki
Article in German by TU Darmstadt press news: https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/einzelansicht_313024.de.jsp
irosalab.com 