11LE13P-7302 Laboratory, Deep Learning for Autonomous Driving

• Prof. Dr. Wolfram Burgard,
• Dr. Daniel Buescher, Abhinav Valada, Johan Vertens, Oier Mees, Andreas Eitel Ayush Dewan

About this course: Self-driving cars represent one of the most exciting advances in modern history. Their impact will go beyond technology, beyond transportation, beyond urban planning to change our daily lives in ways we have yet to imagine. Students who enroll in this course will gain basic theoretical and practical knowledge of deep learning through the applied theme of autonomous driving. The course will have introductory lectures on convolutional neural networks, supervised and semi-supervised learning, self-driving cars and then continues with specialized project work. The areas of focus include scene understanding, state estimation and end-to-end driving. Students will implement their solutions depending on the project on real-world driving platforms: an Audi car with a full suite of sensors including multiple cameras, LiDARS, highly precise GPS and two powerful computers with GPU’s, or an 1/8 scale Audi car with cameras and onboard GPU computing. The students will work in teams of 2-3 and will present their solutions at a poster session and live demonstrations at the end of the semester.

Who is this class for: This course is primarily aimed at students who have experience in programming (Python, C++), basic calculus and knowledge about machine learning (supervised learning problems such as classification, regression, segmentation). Some experience with deep learning (e.g. you attended the deep learning lab last semester) and the Linux toolchain (editor, compiler, linker, debugger) would be beneficial.

Lecture/exercises: Friday, 16.00-18.00 Room: Seminarroom Building 080 / Computer Pools Building 082 Beginning: Friday, April 20, 2018 in Seminarroom Building 080 (You can also join if you have not enrolled) Communications: Join our Slack channel for updates on the course (dl4ad.slack.com) (You can create a userwith your university email address or contact valada@cs.uni-freiburg.de for an invitation) Project Assignment: Please fill in this questionnaire to help us assign the projects:Form Course syllabus: There will be four lectures: self-driving cars, deep learning, pytorch tutorial and advanced deep learning. The lectures will be accompanied by three months of project work in groups of 2-3 students. We offer 6 projects covering different areas. Every group will work on one project throughout the semester and discuss their progress in weekly meetings. Project 1: Semantic free-space estimation. The goal is to segment drivable regions in images and further mark edges of objects intersecting the drivable area into various semantic categories. Methods: fully-convolutional neural networks. Project 2: Semantic 3D object detection. Predicting 3D bounding boxes of road objects and their category from pointclouds. Methods: Faster R-CNN, SSD, pointcloud processing. Project 3: Steering angle prediction. Estimate the current and future trajectory of the car conditioned on the current visual state and speed. Methods: temporal models, recurrent neural networks, LSTMs, gated recurrent units, deep regression. Project 4: End-to-end driving for a small 1/8 scale car. Predict the steering angle and acceleration from visual input and high-level human commands. Methods: conditioned imitation learning, reinforcement learning. Project 5: Vision-based localization in street scenes. Use sensor fusion in combination with deep regression networks. Methods: deep regression, sensor fusion, odometry, state estimation. Project 6: Semantic road segmentation for a small 1/8 scale car. Methods: fully convolutional neural networks, projective geometry, semi-supervised learning. Presentations: Lecture 1: Autonomous Cars Lecture 2: CS231 lecture 5 CS231 lecture 6 Pytorch: First steps MNIST example code Lecture 3: CS231 lecture 9 Lecture 4: CS231 lecture 11 Google Cloud Tutorial Assignments: Exercise 1: Traffic Sign Recognition Final Exercise Poster Template Remarks: The short lectures will be in English. The course is open to both Bachelor students (for their Bachelor project) and Master students (for their lab course). Further material: Support for this course was generously provided by the Google Cloud Platform Education Grant.