Course Description

This course will discuss concepts and algorithms for reinforcement learning (RL) and multi-agent RL (MARL). The goal is for students to understand: (1) key concepts, (2) key algorithms and their implementation, and (3) new topics in the field. Topics include models (MDPs, games) and algorithms (RL, cooperative MARL, and non-cooperative MARL).

Pre-requisites: Probability and statistics (STAT 361, STAT 400, ISE 300, or equivalent) or permission from the instructor.

Course Credits: 4 credit hours.

Learning Outcomes

  • Understand the mathematical fundamentals of key theoretical and algorithmic concepts in RL and MARL
  • Know how to implement and use common RL and MARL algorithms with code
  • Gain experience applying RL and MARL to open-ended research problems
  • Communicate technical information clearly and concisely

Course Content and Tools

Links to course content are at the top of this website.

Announcements and Discussions

Announcements and discussions will be handled on Piazza. Find our Piazza class signup link at: https://piazza.com/illinois/fall2025/ae_598_120258_252834.

We recommend you set up notifications to keep up with announcements. Any questions about concepts, assignments, or course material should be made public to avoid answering the same question multiple times. Feel free to post anonymously to your peers or anonymously to everyone (including instructors) as desired. Messages regarding personal issues (e.g., sickness, leave, individual grades) should be sent privately to the instructor(s).

Textbook and Lectures

Lecture materials will posted to a shared directory on Google Drive.

We will loosely follow various topics from Sutton and Barto’s RL textbook (available for free here) and Albrecht et al.’s MARL textbook (available for free here).

Assignments

Assignments will be distributed using GitHub Classroom—links to assignments will be posted on the Schedule page. If you have not used GitHub, there is a short tutorial available here.

Assignment submissions and grades will be handled on Gradescope.

See the Resources/Assignments page for more details regarding assignment distribution and submissions.

Python

All coding assignments will be done with Python. Python is open-source, widely used, and has a very active support community (e.g., stack overflow). You are expected to already be proficient in Python.

See the Resources page for resources related to coding.

Grading

Homeworks: There will be a few (~2-4) homework assignments, likely containing a mix of theoretical and coding problems.

You are encouraged to work together on homeworks, but each student should prepare and submit their own work. Homework that is viewed as insufficiently distinct to warrant an independent submission will not be given credit, and, depending on the situation, may be submitted as cheating via the FAIR system.

Projects: There will be one project. The project will be open-ended and aims to offer you an opportunity to implement your choice of methods and apply them to a research problem of interest to you.

Literature Reviews: Each student will perform one literature review throughout the semester, where you choose a MARL (or RL) paper of interest and give a presentation discussing the paper to the class. Each literature review presentation will also be moderated by two (other) students.

Drop Pollicy: Your lowest homework grade will be dropped.

Late Policy: Late homework and project submissions will be accepted up to 72 hours after the deadline with the following deductions: -10 points (within 24 hours of the deadline), -15 points (within 48 hours of the deadline), -20 points (within 72 hours of the deadline).

Final Grade Your final grade will calculated from homeworks (40%), projects (40%), and literature reviews (20%). The following grading scale will be used:

Grade Point Range
A [93, 100)
A- [90, 93)
B+ [87, 90)
B [83, 87)
B- [80, 83)
C+ [77, 80)
C [73, 77)
C- [70, 73)
D+ [67, 70)
D [63, 67)
D- [60, 63)
F < 60

Other Information

Time Management

You will earn four credit hours for completing this course. The federal definition of a “credit hour” is an amount of work that reasonably approximates not less than one hour of instruction and two hours of out-of-class student work each week throughout the semester. Therefore, each of you should expect to spend about twelve hours on this course each week, including the time you spend in lecture. If you find yourself spending much less or much more time and are struggling with time management, please ask for help.

Respect and Growth in the Classroom

The effectiveness of our course is dependent upon each of us to create a safe and encouraging learning environment that allows for the open exchange of ideas while also ensuring equitable opportunities and respect for all of us. Everyone is expected to help establish and maintain an environment where students, staff, and faculty can contribute without fear of personal ridicule, or intolerant or offensive language. We ask everyone to be ready to learn and grow in your respect and understanding of others, in addition to your understanding of the course material.

Inclusivity

A feeling of belonging and inclusion is critical to the success and health of our community. The Aerospace Engineering department has a committee called Aero’s Space to Belong. They offer office hours, one-on-one discussion, and a reporting process. If you experience conflict that undermines your or someone else’s feelings of belonging, please consider using these resources: https://aerospace.illinois.edu/diversity/reporting.

Accomodations

Any student with special needs or circumstances requiring accommodation in this course (e.g., disability-related academic adjustments and/or auxiliary aids) is encouraged to contact the instructor and the Disability Resources and Educational Services (DRES) as soon as possible. To contact DRES, you may visit 1207 S. Oak St., Champaign, call 333-4603, e-mail disability@illinois.edu or go to the DRES website. We will ensure these special needs are met.

Additional References