PAL robotics has interviewed Georgia about her career path, working with TIAGo, and her vision for iROSA.
Find the first part of the interview at: https://blog.pal-robotics.com/georgia-chalvatzaki-interview-part-1-starting-in-robotics-tiago-robot-home-robots-and-more/
In this thesis, we will investigate the use of 3D primitive representations in objects using Invertible Neural Networks (INNs). Through INNs we can learn the implicit surface function of the objects and their mesh. Apart from extracting the object’s shape, we can parse the object into semantically interpretable parts. In our work our main focus will be to segment the parts in objects that are semantically related to object affordances. Moreover, the implicit representation of the primitive can allow us to compute directly the grasp configuration of the object, allowing grasp planning.
Interested students are expected to have experience with Computer Vision and Deep Learning, but also know how to program in Python using DL libraries like PyTorch. The thesis will be co-supervised by Despoina Paschalidou (Ph.D. candidate at the Max Planck Institute for Intelligent Systems and the Max Planck ETH Center for Learning Systems). Highly motivated students can apply by sending an e-mail expressing your interest to georgia.chalvatzaki@tu-darmstadt.de, attaching your CV and transcripts.
References:
[1] Paschalidou, Despoina, Angelos Katharopoulos, Andreas Geiger, and Sanja Fidler. “Neural Parts: Learning expressive 3D shape abstractions with invertible neural networks.” arXiv preprint arXiv:2103.10429 (2021).
[2] Karunratanakul, Korrawe, Jinlong Yang, Yan Zhang, Michael Black, Krikamol Muandet, and Siyu Tang. “Grasping Field: Learning Implicit Representations for Human Grasps.” arXiv preprint arXiv:2008.04451 (2020).
[3] Chao, Yu-Wei, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj S. Narang et al. “DexYCB: A Benchmark for Capturing Hand Grasping of Objects.” arXiv preprint arXiv:2104.04631 (2021).
[4] Do, Thanh-Toan, Anh Nguyen, and Ian Reid. “Affordancenet: An end-to-end deep learning approach for object affordance detection.” In 2018 IEEE international conference on robotics and automation (ICRA), pp. 5882-5889. IEEE, 2018.
My talk on “Accelerating Robot Skill Learning with Demonstrations and Models”, that I gave a few days ago at the AI in Robotics Seminar at the University of Toronto is now available online.
In this talk I go through our recent works “Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills” and “Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning” that will appear at ICRA2021.
iROSA research group has received the hessian.AI Connectom fund, which promotes interdisciplinary research in the hessian.AI ecosystem.
This grant will allow researches from iROSA to work along researches from the Ubiquitous Knowledge Processing lab, led Prof. Dr. Gurevych.
Our project will investigate synergies between Robot Learning and Natural Language Processing towards Learning Long-Horizon Tasks by bridging Object-centric Representations to Knowledge Graphs.
AI Newcomer 2021 of the category technical and engineering science is Dr. Georgia Chalvatzaki!
“AI advances robotic research for developing intelligent agents that interact safely and assist humans” – Georgia Chalvatzaki
Article in German by TU Darmstadt press news: https://www.tu-darmstadt.de/universitaet/aktuelles_meldungen/einzelansicht_313024.de.jsp
Grasp planning is one of the most challenging tasks in robot manipulation. Apart from perception ambiguity, the grasp robustness and the successful execution rely heavily on the dynamics of the robotic hands. The student is expected to research and develop benchmarking environments and evaluation metrics for grasp planning. The development in simulation environments as ISAAC Sim and Gazebo will allow us to integrate and evaluate different robotic hands for grasping a variety of everyday objects. We will evaluate grasp performance using different metrics (e.g., object-category-wise, affordance-wise, etc.), and finally, test the sim2real gap when transferring such approaches from popular simulators to real robots.
The student will have the chance to work with different robotic hands (Justin hand, PAL TIAGo hands, Robotiq gripper, Panda gripper, etc.) and is expected to transfer the results to at least two robots (Rollin’ Justin at DLR and TIAGo++ at TU Darmstadt). The results of this thesis are intended to be made public (both the data and the benchmarking framework) for the benefit of the robotics community.
As this thesis is offered in collaboration with the DLR institute of Robotics and Mechatronics in Oberpfaffenhofen near Munich, the student is expected to work in DLR for a period of 8-months for the thesis. On-site work at the premises of DLR can be expected but not guaranteed due to COVID-19 restrictions. A large part of the project can be carried out remotely.
Highly motivated students can apply by sending an e-mail expressing their interest to daniel.leidner@dlr.de and georgia.chalvatzaki@tu-darmstadt.de
Please attach your CV and your transcripts.
References:
[1] Collins, Jack, Shelvin Chand, Anthony Vanderkop, and David Howard. “A Review of Physics Simulators for Robotic Applications.” IEEE Access (2021).
[2] Bekiroglu, Y., Marturi, N., Roa, M. A., Adjigble, K. J. M., Pardi, T., Grimm, C., … & Stolkin, R. (2019). Benchmarking protocol for grasp planning algorithms. IEEE Robotics and Automation Letters, 5(2), 315-322.
Georgia Chalvatzaki gave a talk on Thursday, March 18 at the Learning and Intelligent Systems group of TU Berlin. She presented recent results, but also her vision and ongoing research within the iROSA group and her Emmy Noether research project.
Three papers got accepted in ICRA2021, whose topics will be directly extended in the context of the iROSA project.
Accepted papers:
Tosatto, S.; Chalvatzaki, G.; Peters, J. (2021). Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). See Details BibTeX Reference
Li, Q.; Chalvatzaki, G.; Peters, J.; Wang, Y. (2021). Directed Acyclic Graph Neural Network for Human Motion Prediction, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). See Details BibTeX Reference
Morgan, A.; Nandha, D.; Chalvatzaki, G.; D’Eramo, C.; Dollar, A.; Peters, J. (2021). Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learning, Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). See Details BibTeX Reference
The newly founded Intelligent Robotics for Assistance (iROSA) group, led by Dr. Georgia Chalvatzaki, in cooperation with the Intelligent Autonomous Systems Lab (IAS) at the Technical University of Darmstadt (TU Darmstadt) is seeking a Ph.D. student with a strong interest in the following research topic:
Robot Learning Task and Motion Planning of Long-horizon (mobile) manipulation tasks.
The Ph.D. student will work on a highly interdisciplinary topic at the intersection of machine learning and classical robotics. Following the increasing demand for embodied AI agents that will serve as assistants in houses, workplaces, etc., we will research how intelligent behavior may be acquired by the continual purposeful interaction of an agent with an environment and the induced sensorimotor experience. Our central research question is “How can embodied AI systems, specifically mobile manipulator robots, acquire skills for performing long-horizon assistive tasks in human-inhabited environments?”. As planning for assistive tasks requires impractical computational time, coupling planning with learning methods is key to advancing the state-of-the-art in the field of mobile manipulation. Before the introduction of deep Reinforcement Learning, learning methods were not able to scale well to high-dimensional problems, thus prohibiting their use in real-world problems. iROSA group aims to create mobile manipulation robot assistants with the ability to intelligently acquire their skills, fluently interact with humans through handover tasks, and dynamically adapt their behavior for accomplishing long-horizon household tasks, like the fetch-carry-handover, in human-inhabited environments. For long-horizon planning, we will explore ideas from classic Task and Motion planning, Graph neural networks for combinatorial optimization, Hierarchical Reinforcement Learning, self-supervised Representation Learning, etc.
Students and researchers from the areas of robotics and robotics-related areas including machine learning, control engineering, and computer vision are welcome to apply. The candidates are expected to conduct independent research and at the same time contribute to the research topics listed above. Women and people of underrepresented minority groups are strongly encouraged to apply.
ABOUT THE APPLICANT
Ph.D. position applicants need to have a Master’s degree (high grade required) in a relevant field (e.g., Robotics, Computer Science, Engineering, Statistics & Optimization, Math, and Physics). Expertise in working with real robot systems (including e.g. programming in ROS and sensor data processing) and/or computer vision, deep learning is a big plus. Note that we favor heavily candidates with real robot experience.
THE POSITION
The position is for a 36-month contract. Payment will be according to the German TVL payment scheme.
HOW TO APPLY?
All complete applications submitted through our online application system found at https://www.ias.informatik.tu-darmstadt.de/Jobs/Application will be considered.
The position is planned to start between June 2021 and September 2021 depending on the candidate’s availability.
There is no official deadline, but we will adopt a first-come, first-served policy!
Ph.D. applicants should provide a comprehensive research statement on their research experience and motivation about the Ph.D. topic, a PDF with their CV, degrees (Bachelor’s and Master’s), and grade-sheets, and at least two references who are willing to write a recommendation letter.
Please state clearly how your experience in robotics, computer vision, and machine learning relates to the offered topics in your Research Statement.
Please ensure to include your date of availability for starting the Ph.D. position.
After submitting the application, send a quick notification with the subject line “Ph.D. student applicant <your name>” to Dr. Georgia Chalvatzaki (georgia.chalvatzaki@tu-darmstadt.de) and include your application number in the e-mail.
ABOUT iROSA and IAS
The iROSA group (https://irosalab.com/) is a newly founded group on intelligent robotics for assistance led by Dr. Georgia Chalvatzaki (https://www.ias.informatik.tu-darmstadt.de/Team/GeorgiaChalvatzaki). Georgia, previously a postdoctoral researcher at the Intelligent Autonomous Systems group (IAS) in the Department of Computer Science at TU Darmstadt, has been accepted into the renowned Emmy Noether Programme (ENP) of the German Research Foundation (DFG) in 2021. This project was awarded within the ENP Artificial Intelligence call of the DFG – only 9 proposals out of 91 proposals were selected for funding. It enables outstanding young scientists to qualify for a university professorship by independently leading a junior research group over six years. In her research group iROSA, Dr. Chalvatzaki and her new team will research the topic of “Robot Learning of Mobile Manipulation for Assistive Robotics”. Dr. Chalvatzaki proposes new methods at the intersection of machine learning and classical robotics, taking one step further the research for embodied AI robotic assistants. The research in iROSA proposes novel methods for combined planning and learning for enabling mobile manipulator robots to solve complex tasks in house-like environments, with the human-in-the-loop of the interaction process. The iROSA group has access to two bi-manual manipulator robots TIAGo++ by PAL robotics, a dedicated OptiTrack Motion Capture System, Kinect Azure, and RealSense cameras, a cluster for accelerated computing, etc.
Dr. Chalvatzaki completed her Ph.D. studies in 2019 at the Intelligent Robotics and Automation Lab at the Electrical and Computer Engineering School of the National Technical University of Athens, Greece, with her thesis “Human-Centered Modeling for Assistive Robotics: Stochastic Estimation and Robot Learning in Decision Making.” During her career, she has worked on eight research projects, and she has published more than 35 papers (Google scholar), most of which in top-tier robotics and machine learning venues, e.g., ICRA, IROS, RA-L. Her research at the Computer Science department of TU Darmstadt has been about human-robot collaboration and joint action. In her recent work, she focused on robotic grasping, manipulation, and motion prediction, introducing novel methods for orientation attentive grasp synthesis, accelerated skill learning, and human intention prediction.
The IAS group of TUDa (https://www.ias.informatik.tu-darmstadt.de/) is considered one of the strongest robot learning groups in Europe with expertise ranging from the development of novel machine learning methods (e.g., novel reinforcement learning approaches, policy search, imitation learning, regression approaches, etc.) over semi-autonomy of intelligent systems (e.g., shared control, interaction primitives, human-collaboration during manufacturing) to fully autonomous robotics (e.g., robot learning architectures, motor skill representation acquisition & refinement, grasping, manipulation, tactile sensing, nonlinear control, operational space control, robot table tennis). IAS members are well-known researchers both in the machine learning and the robotics community. The lab collaborates with numerous universities in Germany, Europe, the USA, and Japan as well as companies such as ABB, Honda Research, Franka Emika, and Porsche Motorsport. The iROSA and the IAS lab are located in the city center campus of TU-Darmstadt close to the beautiful Herrngarten park.
ABOUT TU DARMSTADT
TU Darmstadt is one of Germany’s top technical universities and is well known for its research and teaching. It was one of the first universities in the world to introduce electrical engineering programs, and it is Germany’s first fully autonomous university. More information can be found on https://en.wikipedia.org/wiki/Technische_Universit%C3%A4t_Darmstadt
ABOUT DARMSTADT
Darmstadt is a well-known high-tech center with essential activities in spacecraft operations (e.g., through the European Space Operations Centre, the European Organization for the Exploitation of Meteorological Satellites), chemistry, pharmacy, information technology, biotechnology, telecommunications, and mechatronics, and consistently ranked among the Top high-tech regions in Germany. Darmstadt’s important centers for arts, music, and theatre allow for versatile cultural activities, while the proximity of the Odenwald forest and the Rhine valley allows for many outdoor sports. The 33,547 students of Darmstadt’s three universities constitute a significant part of Darmstadt’s 140,000 inhabitants. Darmstadt is located close to the center of Europe. With just 17 minutes driving distance to the Frankfurt airport (closer than Frankfurt itself), it is one of Europe’s best-connected cities. Most major European cities can be reached within less than 2.5h from Darmstadt.
Talk about my research on robot learning for intelligent assistance at the Robot Learning Seminar Series of the Robotics and Embodied AI Lab and Mila, streamed on YouTube.
Abstract: Societal facts like the increase in life expectancy, the lack of nursing staff, the hectic rhythms of everyday life, and the recent unprecedented situation of the Covid-19 pandemic, make the need for intelligent robotic assistants more urgent than ever. Spanning their applications from home-environments to hospitals, workhouses to agricultural development, etc., the embodied AI robotic assistants are in the epicenter of modern robotics and AI research. In this talk, I will go through my research work for developing intelligent methods for such assistive agents. We will draw the big picture of intelligent service robots, and we will specifically focus on sub-problems that I have tackled in the last few years. The main research areas we will cover consider: the perception and recognition of human activities, combining classical methods like tracking with machine learning for extracting useful multi-sensor human-related information for robot action planning and control; algorithms for encoding object-features that allow 6D tracking and grasp-planning; we will discuss methods that can leverage human-centered information for learning intelligent robot behavior using reinforcement learning; and, I will elaborate on our recent work about accelerated policy learning of manipulation tasks both through the effective combination of imitation and reinforcement learning, but also through a novel method for model-predictive policy optimization. While these topics cover only partial aspects of the bigger problem, we will discuss the open research questions on the combination of learning, reasoning, and planning in unstructured environments using mobile manipulator robots. Mobile manipulators are the most emblematic systems to encapsulate the benefits of embodied AI research towards achieving the long-term vision of developing intelligent robotic assistants.
