Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to redesign an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent advances in deep reinforcement learning, and specifically the soft actor critic algorithm. Key to our approach is the possibility of leveraging previously tested morphologies and behaviors to estimate the performance of new candidate morphologies. As such, we can make full use of the information available for making more informed decisions, with the ultimate goal of achieving a more data-efficient co-adaptation (i.e., reducing the number of morphologies and behaviors tested). Simulated experiments show that our approach requires drastically less design prototypes to find good morphology-behaviour combinations, making this method particularly suitable for future co-adaptation of robot designs in the real world.
@inproceedings{pmlr-v100-luck20a,title={Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning},author={Luck, Kevin Sebastian and Amor, Heni Ben and Calandra, Roberto},booktitle={Proceedings of the Conference on Robot Learning},pages={854--869},year={2020},editor={Kaelbling, Leslie Pack and Kragic, Danica and Sugiura, Komei},volume={100},series={Proceedings of Machine Learning Research},month={30 Oct--01 Nov},publisher={PMLR},url={https://proceedings.mlr.press/v100/luck20a.html},}
NeurIPS
Conditional Mutual Information for Disentangled Representations in Reinforcement Learning
Mhairi Dunion, Trevor McInroe, Kevin Sebastian Luck, and 2 more authors
In Thirty-seventh Conference on Neural Information Processing Systems, 30 oct–01 nov 2023
@inproceedings{dunion2023conditional,title={Conditional Mutual Information for Disentangled Representations in Reinforcement Learning},author={Dunion, Mhairi and McInroe, Trevor and Luck, Kevin Sebastian and Hanna, Josiah P. and Albrecht, Stefano V},booktitle={Thirty-seventh Conference on Neural Information Processing Systems},year={2023},url={https://openreview.net/forum?id=EmYWJsyad4},}
NeurIPS
Practical Equivariances via Relational Conditional Neural Processes
Daolang Huang, Manuel Haussmann, Ulpu Remes, and 5 more authors
In Thirty-seventh Conference on Neural Information Processing Systems, 30 oct–01 nov 2023
@inproceedings{huang2023practical,title={Practical Equivariances via Relational Conditional Neural Processes},author={Huang, Daolang and Haussmann, Manuel and Remes, Ulpu and John, S. T. and Clart{\'e}, Gr{\'e}goire and Luck, Kevin Sebastian and Kaski, Samuel and Acerbi, Luigi},booktitle={Thirty-seventh Conference on Neural Information Processing Systems},year={2023},url={https://openreview.net/forum?id=xax5eWeObb},}
AAAI
Co-imitation: learning design and behaviour by imitation
Chang Rajani, Karol Arndt, David Blanco-Mulero, and 2 more authors
In Proceedings of the AAAI Conference on Artificial Intelligence, 30 oct–01 nov 2023
@inproceedings{rajani2023co,title={Co-imitation: learning design and behaviour by imitation},author={Rajani, Chang and Arndt, Karol and Blanco-Mulero, David and Luck, Kevin Sebastian and Kyrki, Ville},booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},volume={37},number={5},pages={6200--6208},year={2023},}
Investigation into Bio-inspired Snake Robot Designs with Co-Adaptation of Morphology and Behaviour
Riccardo Pretto, Sylvia Cressmann, Shivam Chaubey, and 3 more authors
In 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40), 30 oct–01 nov 2024
@inproceedings{pretto_snake,author={Pretto, Riccardo and Cressmann, Sylvia and Chaubey, Shivam and Jansen, Michael Andrew and Kyrki, Ville and Luck, Kevin Sebastian},title={Investigation into Bio-inspired Snake Robot Designs with
Co-Adaptation of Morphology and Behaviour},booktitle={40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40)},year={2024},publisher={Abstract},}
Following Ancestral Footsteps: Co-Designing Morphology and Behaviour with Self-Imitation Learning
Sergio Hernandez-Gutierrez, Ville Kyrki, and Kevin Sebastian Luck
In Seventeenth European Workshop on Reinforcement Learning, 30 oct–01 nov 2024
@inproceedings{hernandezfollowing,title={Following Ancestral Footsteps: Co-Designing Morphology and Behaviour with Self-Imitation Learning},author={Hernandez-Gutierrez, Sergio and Kyrki, Ville and Luck, Kevin Sebastian},booktitle={Seventeenth European Workshop on Reinforcement Learning},year={2024},}
CoRL
Learning Transparent Reward Models via Unsupervised Feature Selection
Daulet Baimukashev, Gokhan Alcan, Kevin Sebastian Luck, and 1 more author
In 8th Annual Conference on Robot Learning, 30 oct–01 nov 2024
@inproceedings{baimukashevlearning,title={Learning Transparent Reward Models via Unsupervised Feature Selection},author={Baimukashev, Daulet and Alcan, Gokhan and Luck, Kevin Sebastian and Kyrki, Ville},booktitle={8th Annual Conference on Robot Learning},year={2024}}
