Description
In this section, we are going to implement an agent which can play the Frozen Lake game (Version 1). You may read about it in the documentation at gymlibrary.dev/environments/toy_text/frozen_lake. We implement the basic SARSA and Q-Learning approaches. You have been given with source files and should only complete it. Attention: In the source files, you only work with module Agent.py . There, you only need to complete ## COMPLETE parts. There are several functions and loops that have been already implemented. Feel free to use them as a reference code to implement the other parts. Preparation To get started open the source files and run the main.py file. In the preliminary form of the provided file, the agent plays for 10 times randomly. Go over the code in both main.py and Agent.py to get familiar with the environment and class Agent() . To run the code you need to have the libraries gymnasium , numpy , tqdm , matplotlib , and pickle installed. To install any of these libraries, you can run 1 pip install in the terminal. Question 1 [3 Points] (Initial Action-Values) Initiate the action-values with zeros. To this end, complete the requested line in method __init__() . You should define self.Q to be a matrix. Question 2 [7 Points] (ϵ-Greedy Policy) Complete the method epsGreedy() which gets the matrix of action values and the state and returns an action based on ϵ-greedy policy. Question 3 [25 Points] (SARSA) Complete the method SARSA() to implement the basic SARSA algorithm, i.e., the one based on TD-0 evaluation (with no eligibility tracing). 1. Train your agent for 1000 episodes and save matrix of action-values as well as the learning curve. 2. Test the trained agent for 100 episodes and print the success_rate . All required parameters are specified in main.py . Attention: Submit the .pkl file, the learning curve, and the final success_rate .
Question 4 [25 Points] (Q-Learning) Complete the method QLearning() to implement the basic Qlearning algorithm. 1. Train your agent for 1000 episodes and save matrix of action-values as well as the learning curve. 2. Test the trained agent for 100 episodes and print the success_rate . All required parameters are specified in main.py . Attention: Submit the .pkl file, the learning curve, and the final success_rate .