Prosody as a Teaching Signal for Agent Learning: Exploratory Studies and Algorithmic Implications

The repository is composed of two parts:

1. wizard_of_oz: a remote platform for emulating online interactive learning scenarios with audio feedback, in which one participant gives audio feedback to the agent, and another participant remotely controls the robot as a response to that feedback. Even though it was designed to emulate an interactive RL scenario with evaluative feedback in a grid-world Robotaxi task, it can be built upon to richer and more diverse scenarios that involve audio or speech feedback.

2. javatari: an application allowing a user to provide Atari game demonstrations with audio as an additional channel.

Citations

If you find our work to be useful in your research, please cite:

  @article{knierim2024prosody,
    title={Prosody as a Teaching Signal for Agent Learning: 
    Exploratory Studies and Algorithmic Implications},
    author={Knierim, Matilda and Jain, Sahil and Han Aydoğan, Murat
    and Mitra, Kenneth and Kush Desai and Saran, Akanksha and Baraka, Kim},
    journal={ACM International Conference on Multimodal Interaction (ICMI)},
    year={2024}
  }