Learning in Sparse Reward & Imperfect Information Games - IEEE CoG2022 Tutorial II

5 views
Download
  • Share

Tutorial II of 2022 IEEE Conference on Games given by Junliang Xing and Kai Li, with the title of "Learning in Sparse Reward and Imperfect Information Games."

In recent years, artificial intelligence has made many breakthroughs, from playing Atari games to learning complex robotic manipulation tasks. However, in many real-world scenarios, an agent still faces challenges from sparse rewards and imperfect information. Sparse reward refers to a reward function that is zero in most of its domain and only gives positive values to very few states.

It is difficult for an agent to learn effective policies in sparse reward games since it will not get any feedback about whether its instantaneous actions are good or bad. Moreover, the presence of imperfect information in games, where an agent does not fully know the state of the world, makes learning more difficult since learning in such imperfect information games requires reasoning under uncertainty about other agents’ private information.

This tutorial focuses on commonly used approaches for learning in sparse reward and imperfect information games. The first part of the tutorial will discuss learning in sparse reward games, in which we will cover prediction error-based, novelty-based, and information gain-based methods. Also introduced are the latest results of our research group, such as self-navigation-based, potential-based, and influence-based learning algorithms.

The second part of the tutorial focuses on learning in imperfect information games. First introduced are the formal definition of imperfect information games. Then, discussion on regret-based (CFR) and population-based methods (PSRO) to learn fixed optimal worst-case policies.

Finally, the speakers introduce opponent modeling-based methods to learn adaptive policies. Also discussed are the unsolved problems and the directions for future research. This tutorial is intended to inspire and motivate attendees to continue learning and contributing to the current development in this field.

Tutorial II of 2022 IEEE Conference on Games given by Junliang Xing and Kai Li, with the title of "Learning in Sparse Reward and Imperfect Information Games."

In recent years, many breakthroughs have been made in artificial intelligence, from playing Atari games to...

Advertisment

Advertisment