IERG 5350 Assignment 1: Tabular Reinforcement Learning

Due: 11:59 pm, September 30, 2020

Welcome to the first assignment of this course! As this course assignment is still being polished, we are gratitude to your suggestion on how to improve. If you find anything confusing or incorrect no matter in codes or comments or documents, feel free to tell us via email or Github issues (the latter is preferable).

The objective of the Assignment 1 is to

1. make you get familiar with the basic framework of reinforcement learning (RL), such as the agent-environment interaction loops.

2. implement the basic tabular RL algorithms to solve a toy problem.

We will first go through the installation and environment setup. This assignment uses jupyter notebook as the platform for you to code. So you need to get familiar. A quicktutorial on jupyter notebook is here. In the first section of notebook, we take a glance of the environment and RL pipeline. In the second and third sections you are required to implement the contents of tabular RL. This document draw you your path to finish this assignment. We left sufficient information in the jupyter notebook so please walk through the file: IERG 6130 Assignment 1.ipynb.

Step 1: Setup your environment

The starter codes is at: https://github.com/cuhkrlcourse/ierg5350-assignment, go through the document at the root of this repostory (namely, README.md) for environment setup procedures. You need to prepare such packages:

1. Python 3
2. Jupyter Notebook
3. Gym
4. Numpy

If you find anything confusing, feel free to open an issue at Github. We will response to you as soon as possible. If you get stuck by some strange errors, check the Github repo since it may be fixed by our latest commits.

Step 2: Finish the notebook

Open a shell and cd to current directory ierg5350-assignment/assignment1 and start a jupyter notebook server via jupyter notebook.

Open your browser and then open the IERG 5350 Assignment 1.ipynb

We left many [TODO] in the file. Following the instructions in the notebook and finish all [TODO].

Make sure your code can be run completely bug-free. Our staff will run the codes by Restart & Run All so make sure nothing stop the running.

Step 3: Submit your work

(Update: Please do not push your solution to public!)

Following the procedure to submit your work:

1. Before submitting, remember to fill your name and student ID into the table at the top of the file.
2. Run your codes in sequential manner, that is, run it by Restart & Run All. Remember to keep everything intact.
3. Generate the PDF file via File / Download As / pdf to the assignment1 directory. (In case the written jupyter notebook can not be shown in github)
4. Zip and upload your solution to arbitrary cloud storage and create permanent and accessible link.
5. Send the source code link AND the PDF file in your github repo to email: cuhkrlcourse@googlegroups.com with title in pattern "2020fall-IERG5350-hw1-{name}-{id}".format(name="john", id="123456") for instance: 2020fall-IERG5350-hw1-john-123456 We will NOT reply you a recipt email.

---

2020-2021 Term 1, IERG 5350: Reinforcement Learning. Department of Information Engineering, The Chinese University of Hong Kong. Course Instructor: Professor ZHOU Bolei. Assignment author: PENG Zhenghao, SUN Hao, ZHAN Xiaohang.