Course Outline
1. Introduction to Deep Reinforcement Learning
- Defining Reinforcement Learning.
- Distinguishing between Supervised, Unsupervised, and Reinforcement Learning.
- Exploring DRL applications in 2025, spanning robotics, healthcare, finance, and logistics.
- Comprehending the agent-environment interaction loop.
2. Reinforcement Learning Fundamentals
- Markov Decision Processes (MDP).
- Key concepts: State, Action, Reward, Policy, and Value functions.
- Balancing the exploration vs. exploitation trade-off.
- Monte Carlo methods and Temporal-Difference (TD) learning.
3. Implementing Basic RL Algorithms
- Tabular methods: Dynamic Programming, Policy Evaluation, and Iteration.
- Q-Learning and SARSA algorithms.
- Epsilon-greedy exploration strategies and decay techniques.
- Setting up RL environments using OpenAI Gymnasium.
4. Transition to Deep Reinforcement Learning
- Limitations associated with tabular methods.
- Leveraging neural networks for function approximation.
- Architecture and workflow of the Deep Q-Network (DQN).
- Understanding experience replay and target networks.
5. Advanced DRL Algorithms
- Double DQN, Dueling DQN, and Prioritized Experience Replay.
- Policy Gradient Methods, including the REINFORCE algorithm.
- Actor-Critic architectures (A2C, A3C).
- Proximal Policy Optimization (PPO).
- Soft Actor-Critic (SAC).
6. Working with Continuous Action Spaces
- Challenges inherent in continuous control.
- Utilizing DDPG (Deep Deterministic Policy Gradient).
- Twin Delayed DDPG (TD3).
7. Practical Tools and Frameworks
- Using Stable-Baselines3 and Ray RLlib.
- Logging and monitoring via TensorBoard.
- Hyperparameter tuning for DRL models.
8. Reward Engineering and Environment Design
- Reward shaping and balancing penalties.
- Concepts of Sim-to-real transfer learning.
- Creating custom environments in Gymnasium.
9. Partially Observable Environments and Generalization
- Managing incomplete state information (POMDPs).
- Memory-based approaches utilizing LSTMs and RNNs.
- Enhancing agent robustness and generalization capabilities.
10. Game Theory and Multi-Agent Reinforcement Learning
- Introduction to multi-agent environments.
- Dynamics of cooperation vs. competition.
- Applications in adversarial training and strategy optimization.
11. Case Studies and Real-World Applications
- Autonomous driving simulations.
- Dynamic pricing and financial trading strategies.
- Robotics and industrial automation.
12. Troubleshooting and Optimization
- Diagnosing unstable training processes.
- Addressing reward sparsity and overfitting.
- Scaling DRL models across GPUs and distributed systems.
13. Summary and Next Steps
- Recap of DRL architecture and key algorithms.
- Industry trends and research directions, such as RLHF and hybrid models.
- Additional resources and recommended reading materials.
Requirements
- Strong proficiency in Python programming.
- Solid understanding of Calculus and Linear Algebra.
- Fundamental knowledge of Probability and Statistics.
- Experience in building machine learning models using Python alongside NumPy or TensorFlow/PyTorch.
Target Audience
- Developers keen on Artificial Intelligence and intelligent systems.
- Data Scientists investigating reinforcement learning frameworks.
- Machine Learning Engineers specializing in autonomous systems.
Testimonials (3)
I really liked the end where we took the time to play around with CHAT GPT. The room was not set up the best for this- instead of one large table a couple of small ones so we could get into small groups and brainstorm would have helped
Nola - Laramie County Community College
Course - Artificial Intelligence (AI) Overview
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It felt like we were going through directly relevant information at a good pace (i.e. no filler material)
