We are excited to announce the acceptance of the article “Large Language Model-Based Robot Task Planning from Voice Command Transcriptions” at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025). This work was developed within euROBIN’s research activities on service robotics (WP2), focusing on enabling robots to perform domestic tasks from natural language instructions.
One of the central challenges in building a General Purpose Service Robot (GPSR) lies in interpreting human commands, which often contain speech recognition errors and incomplete information. This paper presents an end-to-end pipeline that leverages a Large Language Model (LLM) to directly translate instruction transcripts into coherent and context-aware action plans. By integrating environmental information into the input, the system is able to generate more efficient task executions.
Key highlights of the paper include:
-
An LLM-based approach to bridge natural language instructions and robot task planning
-
Integration of environmental context for improved efficiency
-
Validation through a simulator based on Generalized Stochastic Petri Nets, with a success rate of ~55% on the ALFRED dataset, including unseen environments
-
Successful real-world deployment at RoboCup 2024 in Eindhoven, achieving 2nd place in the GPSR task
You can access the full paper here: https://afonsocerto.com/files/iros2025_llm_robot_planning.pdf
Congratulations to the authors Afonso Certo, Bruno Martins, Carlos Azevedo, and Pedro U. Lima for this significant contribution to advancing natural language understanding in robotics!