Offered topics

Disclaimer: If you want to propose any thesis topic that falls in the general area of the listed research directions, contact directly Prof. Chalvatzaki, and discuss a topic that suits your interests:

Robotic Perception for manipulation and interaction;
Robot learning for embodied agents like mobile manipulation;
Graph Neural Network representations for Robot Learning
Uncertainty-aware robot learning for (mobile) manipulation
Human behavior understanding, and uncertainty quantification;
Human-Robot Interaction;
Long-horizon reasoning, with a focus on combining Graph-based Reinforcement Learning and Planning.

We offer these current topics directly for Bachelor and Master students at TU Darmstadt. Note that we cannot provide funding for any of these theses projects.

We highly recommend that you do robotics and machine learning lectures (Robot Learning, Statistical Machine Learning, Reinforcement Learning, Grundlagen der Robotik, Probabilistic Graphical Models and/or Deep Learning) before applying for a thesis. Even more important to us is that you followed our Intelligent Robotic Manipulation Seminar and our Project Lab, or that you followed both Robot Learning: Integrated Project, Part 1 (Literature Review and Simulation Studies) and Part 2 (Evaluation and Submission to a Conference), before doing a thesis with us.

When you contact the advisor, it would be nice if you could mention (1) WHY you are interested in the topic (dreams, parts of the problem, etc.), and (2) WHAT makes you special for the projects (e.g., class work, project experience, special programming or math skills, prior work, etc.). Supplementary materials (CV, grades, etc.) are highly appreciated. Of course, such materials are not mandatory, but they help the advisor to see whether the topic is too easy, just about right, or too hard for you.

FOR FB16+FB18 STUDENTS: Students from other departments at TU Darmstadt (e.g., ME, EE, IST), you need an additional formal supervisor who officially issues the topic. Please do not try to arrange your home dept advisor by yourself but let the supervisor get in touch with that person instead!

Thema 1: Hyperbolic Graph Convolutional Neural Networks for Robot System Identification

Scope: Master thesis
Advisor: Georgia Chalvatzaki
Start: ASAP

Topic: This thesis will investigate the use of novel approaches in topological representation learning, namely using Graph Neural Networks (GNNs). GNNs have shown increasing applicability to learning Riemannian and Hyperbolic manifold structures. Robotic kinematic chains can naturally be represented in a Riemannian manifold, using as metric the kinetic energy of the robots’ configurations. In this thesis, we will investigate the core components of GNNs for Manifold structure learning, with an application to robot structure kinematic (SE3) autoencoding and the task of system identification (learning of dynamics). The tasks of this thesis require an extensive literature review, and an investigation of possible extensions of current promising methods in the field of robotics.

Requirements: The applicant should have a good mathematical background, and willingness to study topics related to differential geometry and topology, alongside machine learning methods for graph neural networks. Knowledge of Python is necessary. Knowledge of PyTorch (optional).

References:

[1] Sanchez-Gonzalez A, Heess N, Springenberg JT, Merel J, Riedmiller M, Hadsell R, Battaglia P. Graph networks as learnable physics engines for inference and control. InInternational Conference on Machine Learning 2018 Jul 3 (pp. 4470-4479). PMLR.

[2] Liu Q, Nickel M, Kiela D. Hyperbolic graph neural networks. Advances in Neural Information Processing Systems. 2019;32.

[3] Di Giovanni F, Luise G, Bronstein M. Heterogeneous manifolds for curvature-aware graph embedding. arXiv preprint arXiv:2202.01185. 2022 Feb 2.

[4] Chami I, Ying Z, Ré C, Leskovec J. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems. 2019;32.

Thema 2: Combining Deep Reinforcement Learning and 3D Vision for Dual-arm Robotic Tasks

Scope: Master thesis
Advisor: SnehalJauhri
Added: Novermber 10, 2022
Start: ASAP
Topic (in detail): Attach:Theses/OpenTopics/irosa_master_thesis_doc.pdf

Topic (in brief): Recent breakthroughs in Deep Reinforcement Learning (RL) have led to an increased deployment of learning-based methods in robotics. Nevertheless, RL for robotics has been limited to simple setups that assume perfect knowledge about the robot’s environment.
Recent work at the iRosa lab [1] (irosalab.com/rlmmbp) has successfully utilized Deep RL for performing mobile manipulation tasks (i.e. picking and placing objects using the robot arm while moving using the wheeled base of the robot). However, even in these experiments, the robot just used one out of its two arms, and the method assumed perfect perception of the environment.
In this thesis, we aim to build on advances in 3D Vision [2] (stanford.edu/~rqi/pointnet) and combine them with Deep RL to learn using real-world, imperfect 3D information such as point-clouds or occupancy grids. We also aim to solve the more challenging problem of dual-arm mobile manipulation instead of just using a single arm [3].
Note that this thesis builds upon a successful research paper. Therefore, a good starting point in terms of code and theory exists.

Highly motivated students can apply by sending an e-mail expressing their interest to Snehal Jauhri (email: snehal.jauhri@tu-darmstadt.de), attaching your letter of motivation and possibly your CV.

Requirements: Enthusiasm, ambition, and a curious mind go a long way. There will be ample supervision provided to help the student understand basic as well as advanced concepts. However, prior knowledge of reinforcement learning, robotics, and Python programming would be a plus.

References:
[1] S. Jauhri, J. Peters, and G. Chalvatzaki, “Robot learning of mobile manipulation with reachability behavior priors,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8399–8406, 2022.
[2] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 652–660.
[3] K. S. Luck and H. B. Amor, “Extracting bimanual synergies with reinforcement learning,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 4805–4812.

Thema 3: On designing reward functions for robotic tasks

Scope: Bachelor’s thesis/Master’s thesis
Advisor: Davide Tateo, Georgia Chalvatzaki
Start: ASAP

Topic: Defining a proper reward function to solve a robotic task is a complex and time-consuming process. Reinforcement Learning algorithms are sensitive to reward function definitions, and an improper design of the reward function may lead to suboptimal performance of the robotic agent, even in simple low-dimensional environments. This issue makes it complex to design novel reinforcement learning environments, as the reward tuning procedure takes too much time and leads to overcomplicated and algorithm-specific reward functions.

The objective of this thesis is to study and develop a set of guidelines for building Reinforcement Learning environments representing robotics simulated tasks. We will analyze in-depth the impact of different types of reward functions on very simple tasks for continuous control such as navigation, manipulation, and locomotion. We will consider how the state space affects learning (e.g., dealing with rotations) and how we should deal with these issues in a standard Reinforcement Learning setting. Furthermore, we will verify how to design a reward function that leads to a policy producing smooth actions, to minimize the issues of the sim-to-real transfer of the learned behavior.

Requirements

Curriculum Vitae (CV);

A motivation letter explaining the reason for applying for this thesis and academic/career objectives.

Minimum knowledge

Good Python programming skills

Basic knowledge of Reinforcement Learning.

Preferred knowledge

Knowledge of the PyTorch library;

Knowledge of the MuJoCo and PyBullet libraries.

Knowledge of the MushroomRL library.

Accepted candidate will

Port some classical MuJoCo/PyBullet locomotion environments into MushroomRL;

Design a set of simple manipulation tasks using PyBullet;

Design a set of simple navigation tasks;

Analyze the impact of different reward function definitions in these environments;

Verify if the insights coming from the simple tasks still holds in more complex (already existing) environments;

(Optionally) Port some classical MuJoCo/PyBullet locomotion environments into MushroomRL.

Thema 4: Adaptive Human-Robot Interactions with human Trust maximization

Scope: Master thesis
Advisor: Kay Hansel,Georgia Chalvatzaki
Start: April 2023

Topic: Building trust between humans and robots is a major goal of Human-Robot Interaction (HRI). Usually, trust in HRI has been associated with risk aversion: a robot is trustworthy when its actions do not put the human at risk. However, we believe that trust is a bilateral concept that governs the behavior and participation in the collaborative tasks of both interacting parties. On the one hand, the human has to trust the robot about its actions, e.g., delivering the requested object, acting safely, and interacting in a reasonable time horizon. On the other hand, the robot should trust the human regarding their actions, e.g., have a reliable belief about the human’s next action that would not lead to task failure; a certainty in the requested task. However, providing a computational model of trust is extremely challenging.

Therefore, this thesis explores trust maximization as a partially observable problem, where trust is considered as a latent variable that needs to be inferred. This consideration results in a dual optimization problem for two reasons: (i) the robot behavior must be optimized to maximize the human’s latent trust distribution; (ii) an optimization of the human’s prediction model must be performed to maximize the robot’s trust. To address this challenging optimization problem, we will rely on variational inference and metrics like Mutual Information for optimization.
Highly motivated students can apply by sending an e-mail expressing your interest to kay.hansel@tu-darmstadt.de, attaching a letter of motivation and possibly your CV.

Requirements:

Good knowledge of Python and/or C++;
Good knowledge in Robotics and Machine Learning;
Good knowledge of Deep Learning frameworks, e.g, PyTorch

References:
[1] Xu, Anqi, and Gregory Dudek. “Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations.” ACM/IEEE HRI, IEEE, 2015;
[2] Kwon, Minae, et al. “When humans aren’t optimal: Robots that collaborate with risk-aware humans.” ACM/IEEE HRI, IEEE, 2020;
[3] Chen, Min, et al. “Planning with trust for human-robot collaboration.” ACM/IEEE HRI, IEEE, 2018;
[4] Poole, Ben et al. “On variational bounds of mutual information”. ICML, PMLR, 2019.

Thema 5: Causal inference of human behavior dynamics for physical Human-Robot Interactions

Scope: Master Thesis
Advisor: Georgia Chalvatzaki, Kay Hansel Start: April 2023

Topic: In this thesis, we will study and develop ways of approximating an efficient behavior model of a human in close interaction with a robot. We will research the extension of our prior work on the graph-based representation of the human into a method that leverages multiple attention mechanisms to encode relative dynamics in the human body. Inspired by methods in causal discovery, we will treat the motion prediction problem as such. In essence, the need for a differentiable and accurate human motion model is essential for efficient tracking and optimization of HRI dynamics. You will test your method in the context of motion prediction, especially for HRI tasks like human-robot handovers, and you could demonstrate your results in a real-world experiment.

Highly motivated students can apply by sending an e-mail expressing your interest to georgia.chalvatzaki@tu-darmstadt.de, attaching a letter of motivation and possibly your CV.

Minimum knowledge

Good knowledge of Python and/or C++;
Good knowledge of Robotics;
Good knowledge of Deep Learning frameworks, e.g, PyTorch

References

Li, Q., Chalvatzaki, G., Peters, J., Wang, Y., Directed Acyclic Graph Neural Network for Human Motion Prediction, 2021 IEEE International Conference on Robotics and Automation (ICRA).
Löwe, S., Madras, D., Zemel, R. and Welling, M., 2020. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006.10833.
Yang, W., Paxton, C., Mousavian, A., Chao, Y.W., Cakmak, M. and Fox, D., 2020. Reactive human-to-robot handovers of arbitrary objects. arXiv preprint arXiv:2011.08961.