The DSF Virtual Robotics Lab: Scilab-RL

 

Our research involves many theoretical and practical aspects of embodied artificial agents. The concept of embodiment is of particular importance to address research questions related to cognitive science, that go beyond the mere processing of data, as is the case in approaches based on foundation models like ChatGPT.

In Data Science, Machine Learning, and AI, typical manifestations of embodied agents are robots. However, real physical robots are very time-consuming to maintain due to difficult and complex engineering and hardware issues. Since the research tackled by our institute members is focused on the algorithmic and theoretical details that underlay embodiment, we efficiently avoid hardware and focus on virtual robots instead. This enables us to perform experiments much faster, and to spare the funds and the time that would be consumed by robotic hardware. It also makes it possible to work with non-robotic virtual agents, such as realistic biological agents with muscle models, thus giving us much more freedom in our experiment design.

Our virtual robotics lab is implemented as the freely available experimentation platform Scilab-RL. The platform is a well-documented and well-integrated collection of state-of-the-art algorithms and virtual environments based on reinforcement learning and other sense-plan-act-based control approaches. In particular, it integrates the following collection of features:

  • Easy to get started, in particular for students and researchers that are new to RL.
  • A rich collection of experimentation environments based on MuJoCo, MyoSuite, and other physics engines with high-quality 3d-rendered visual output.
  • A rich collection of state-of-the-art reinforcement learning algorithms, including DQN, SAC, DDPG, TD3, PPO, and others. Thanks to the maintainers of Stable Baselines 3!
  • An excellent documentation for generating new research results, e.g., by extending existing implementations of RL algorithms.
  • A standardized hyperparameter management and optimization procedure based on Optuna.
  • Rich online data visualization features, e.g., to render metric values like rewards, action signals, and Q-values over time.

 

 

Together, these features enable us, researchers and students working with us, to efficiently research the foundations of data science, machine learning, and AI. For more details, please consider the GitHub site of Scilab-RL and feel free to try out our lab right away.