About this course
Bring AI into the physical world through perception, planning, and control for robots and autonomous systems.
Course format. Thirteen weeks, four contact hours each: a two-hour lecture (concepts and theory) and a two-hour practice session. The course is project-based; teams carry one running project end to end and present it three times, in weeks 5, 8, and 13.
What you will build
Built a full perceive-plan-act autonomy stack in Python with ROS 2, PyBullet, and PyTorch, fusing sensors with Kalman and particle filters, planning with A-star and RRT, and tracking trajectories with model-predictive and reinforcement-learning controllers under sim-to-real domain randomization.
Expected outcomes
- Derive rigid-body kinematics, transforms, and state representations for robots
- Formalize probabilistic state estimation and the Bayes filter
- Derive the Kalman filter and particle filter for sensor fusion
- Implement motion planning with sampling-based and graph-search algorithms
- Derive feedback control including PID and the linear-quadratic regulator
- Apply reinforcement learning to continuous robot control
- Build perception pipelines fusing camera, lidar, and inertial sensors
- Analyze the sim-to-real gap and domain randomization strategies
- Evaluate autonomy with task success, safety, and robustness metrics
- Deploy an embodied agent in simulation with a real-transfer path
Key topics
- Perception & sensor fusion
- Motion planning
- Control & RL for robotics
- Simulation-to-real transfer
Theoretical foundations
The concepts and results this course rests on.
- rigid-body kinematics, transforms, and robot state representation
- the Bayes filter and the recursive predict-update cycle
- the Kalman filter and the particle filter for sensor fusion
- configuration-space planning with A-star and rapidly-exploring random trees
- PID control and the linear-quadratic regulator
- the discrete-time Riccati equation and model-predictive control
- policy-gradient reinforcement learning for continuous control
Prerequisites
This is a Year-3 course. It assumes the mandatory CS core: data structures and algorithms, operating systems, computer networks, databases, software engineering, and the core mathematics (linear algebra, probability and statistics, calculus, discrete mathematics). It additionally requires the specific prior courses listed below.
Course-specific prerequisites:
- Machine Learning and basic reinforcement learning
- Linear algebra
- Calculus and basic physics
Weekly schedule 13 weeks · lecture + practice
Embodied foundations
Wk 1
Robots, frames, and kinematics
LectureWe cover rigid-body transforms, coordinate frames, forward kinematics, and robot state representation.
PracticeSet up a robot simulator and command joint and base motion.
ProjectChoose the embodied task and simulator for the running project.
Estimation
Wk 2
Probabilistic state estimation
LectureWe derive the Bayes filter, the Markov assumption, and the recursive predict-update cycle.
PracticeImplement a discrete Bayes filter for robot localization.
ProjectAdd a basic state estimator to the project robot.
Wk 3
Kalman filtering
LectureWe derive the Kalman filter and its extended form for nonlinear Gaussian estimation.
PracticeImplement a Kalman filter for fusing noisy position and velocity sensors.
ProjectUpgrade estimation with a Kalman filter for sensor fusion.
Wk 4
Particle filters and SLAM
LectureWe derive the particle filter, importance sampling, and the basics of simultaneous localization and mapping.
PracticeImplement a particle filter for global localization in a known map.
ProjectAdd robust localization to the project robot.
Perception
Wk 5
Perception and sensor fusionPresentation
LectureWe cover camera and lidar sensing, point clouds, calibration, and multi-sensor fusion.
PracticeTeam presentation: each team defends its task specification and metrics.
ProjectLock the specification and build a perception stage fusing sensors.
Planning
Wk 6
Graph-search motion planning
LectureWe derive configuration space, A-star, Dijkstra, and the heuristics behind optimal graph search.
PracticeImplement A-star planning on an occupancy grid.
ProjectAdd grid-based path planning to the robot.
Wk 7
Sampling-based planning
LectureWe cover RRT, RRT-star, probabilistic roadmaps, and planning in high-dimensional configuration spaces.
PracticeImplement RRT and compare path quality with graph search.
ProjectExtend planning to sampling-based methods for complex spaces.
Control
Wk 8
Feedback controlPresentation
LectureWe derive PID control, stability, and the linear-quadratic regulator as optimal feedback.
PracticeTeam presentation: interim demo of the robot navigating with estimation and planning.
ProjectAdd a feedback controller to track planned trajectories.
Wk 9
Optimal and model-predictive control
LectureWe derive the LQR, the discrete-time Riccati equation, and receding-horizon model-predictive control.
PracticeImplement an MPC controller and compare with PID tracking.
ProjectUpgrade control with model-predictive trajectory tracking.
Learning control
Wk 10
Reinforcement learning for control
LectureWe adapt policy-gradient and actor-critic methods to continuous action spaces for robot control.
PracticeTrain an RL policy for a locomotion or manipulation task in simulation.
ProjectAdd a learned control policy to the robot.
Sim-to-real
Wk 11
Bridging the reality gap
LectureWe analyze the sim-to-real gap, domain randomization, system identification, and robustness.
PracticeApply domain randomization and test policy robustness across perturbed dynamics.
ProjectMake the learned policy robust to a sim-to-real transfer.
Integration
Wk 12
Full autonomy stack
LectureWe integrate perception, estimation, planning, and control into one autonomy pipeline and discuss safety.
PracticeConnect all modules into an end-to-end autonomous behavior.
ProjectIntegrate the full perceive-plan-act stack on the robot.
Capstone
Wk 13
Final defensePresentation
LectureWe synthesize estimation, planning, and control theory and survey open problems in embodied AI.
PracticeTeam presentation: final demo with task metrics and an oral defense of design choices.
ProjectDeliver the complete autonomous embodied agent with evaluation results.
AI tools in this course.
Students use AI assistants to generate and refactor ROS 2 nodes, PyBullet and MuJoCo environment code, and Kalman, particle-filter, A-star, RRT, and MPC implementations, vibe-coding the perceive-plan-act stack. They prompt AI to synthesize sensor-noise models and domain-randomization configs, wire message passing between modules, and generate tests for filter convergence and planner correctness. AI also helps analyze trajectory plots, estimation error, and sim-to-real failures to explain why a controller lost tracking.
Student project
Teams build one autonomous embodied agent in simulation, layering perception, probabilistic estimation, motion planning, and control into a full perceive-plan-act stack. The project advances from classical estimation and planning into learned control and a sim-to-real robustness study, backed by the control and estimation theory taught each week.
Requirements
- Build a working system, not a set of disconnected exercises.
- Be original: a new system that solves a real problem, not a re-implementation of a tutorial or course demo.
- Show real depth: real data, real users or realistic load, and engineering trade-offs that are measured rather than assumed.
- Carry one running project from specification to a deployed, defensible result across the whole term.
- Work in a team of three or four and defend the design at each of the three presentations (weeks 5, 8, and 13).
Example projects
Autonomous mobile robot navigationRobotic arm pick-and-placeQuadruped locomotion controllerSelf-driving car in simulationDrone autonomous flight and landingWarehouse multi-robot coordinationUnderwater or aerial inspection robotIndoor delivery robot with SLAM
Assessment & grading
Grading is project-based, with no written exam. Teams of three or four present one running project three times.
| Component | What it covers | Weight |
|---|---|---|
| Project · Specification | Presentation 1 (week 5): problem, objectives, and architecture | 20% |
| Project · Interim | Presentation 2 (week 8): the working system demonstrated live | 30% |
| Project · Final | Presentation 3 (week 13): end-to-end demo with oral defense | 50% |
Tools & platforms
- ROS 2: robotics middleware and tooling
- Gazebo: physics-based robot simulation
- PyBullet: fast rigid-body simulation
- NVIDIA Isaac Sim: high-fidelity robot simulation
- MuJoCo: contact-rich physics for control
- Gymnasium: reinforcement-learning environments
- Stable-Baselines3: RL algorithm implementations
- Open3D: point-cloud and 3D processing
- OpenCV: camera perception
- NumPy: numerical and linear-algebra computation
- Matplotlib: trajectory and estimation visualization
- PyTorch: learned perception and control policies
Free online courses
Existing free, video-based courses this course can build on, for self-study or as a teaching basis.
- UniversityRobotic Manipulation (MIT 6.4210/6.4212), Russ Tedrake
- UniversityUnderactuated Robotics (MIT 6.832), Russ Tedrake
In Hebrew · בעברית
- Ariel University (Campus IL)רובוטים אוטונומיים (Autonomous Robots)
Primary literature
Seminal works to read for graduate-level depth.
References
Books and resources link to an online or publisher page.
- TextbookProbabilistic Robotics
- TextbookPlanning Algorithms
- TextbookReinforcement Learning: An Introduction, 2nd edition
- PaperProximal Policy Optimization Algorithms
- TextbookProbabilistic Machine Learning: An Introduction
- DocumentationROS 2 Documentation
- TextbookDive into Deep Learning
Role in each concentration
| Concentration | Role |
|---|---|
| Intelligent Software Systems | Elective |
| Networking & Cyber Security | Elective |
| AI & Robotics | Core · Semester 2 |
| AI and Quantum Computing for Finance | Elective |
| Immersive Systems & Game Development | Core · Semester 2 |
| Defense Technologies & Autonomous Systems | Core · Semester 2 |