Self-Driving Cars: Perception, Localization, and Control
Build the real numerical core of an autonomous vehicle: motion models, controllers, Kalman filters, camera geometry, and planners, all running in your browser.
A self-driving car answers three questions, several times a second, forever: where am I, what is around me, and what do I do next. This course takes each of those questions down to the arithmetic that answers it, and has you write that arithmetic yourself.
This is the deep track. Fourteen modules take you from the taxonomy of autonomy through vehicle dynamics, classical control, state estimation, camera geometry, learned perception, and hierarchical motion planning, and then close the loop by assembling the whole stack.
What makes this course different
- You write the real algorithms. Nine labs run live Python with NumPy in your browser, with no install and no simulator to configure. You implement the kinematic bicycle model, a PID speed controller with anti-windup, pure pursuit steering, a least squares sensor calibration, a Kalman filter, GNSS and IMU fusion, camera projection, stereo depth, IoU with non-maximum suppression, an occupancy grid, and A-star route search. An AI tutor reviews what you wrote and coaches you toward a correct, idiomatic solution.
- Interactive, not passive. Steer a bicycle model and watch the turn radius change, drag PID gains and see the step response overshoot, widen the process noise and watch a Kalman belief drift and snap back, cast LIDAR beams into an occupancy grid, and race A-star against Dijkstra on a road network.
- Two frameworks that carry the whole course. The SENSE Loop (Sense, Estimate, Notice, Select, Execute) is the spine every module hangs from. The STEER Check (Stability, Tracking error, Effort, Edge cases, Ride quality) is the discipline you apply to every controller, estimator, and planner you build.
- Honest about what is hard. Why level 3 autonomy is the most dangerous rung. Why you cannot prove safety by driving. Why an Extended Kalman Filter that diverges becomes confidently wrong and cannot recover. Why stereo depth error grows with the square of distance.
What you will be able to do
- Model vehicle motion and choose the right model for the speed and manoeuvre
- Design, tune, and debug longitudinal and lateral controllers, including the failure modes that bite in practice
- Build estimators that fuse noisy sensors into a pose with an honest uncertainty
- Reason about camera geometry, calibration, stereo depth, features, detection, and segmentation
- Plan routes, behaviours, and smooth collision-free trajectories inside real comfort and friction limits
- Read an autonomous vehicle architecture diagram and know what every block does and how it fails
Who this is for
Engineers, developers, and technically-minded people who want to understand autonomous vehicles properly rather than at press-release depth. Comfort with Python and secondary-school trigonometry is enough. Every symbol is defined the first time it appears, and the linear algebra you need is built up as you go.
Machine Learning Basics is recommended before Modules 12 and 13, which use convolutional networks for detection and segmentation. Everything else is self-contained. Deep Learning Specialization goes deeper on the perception networks if you want it.
Recommended first
Machine Learning Basics
A gentle, no-math introduction to how machines learn from examples: your warm-up before deep learning.