University of Texas at Arlington - Fall 2010
CSE 4392/5369
Special Topics in
Vision-based Robot Sensing, Localization and Control
1 - Course Overview
2 - Course Material, Topics and Goals
3 - Course Assignments, Exam and Policies
4 - Course/University Policies and Services
|
|
|
| Faculty: |
| Dr. Gian Luca Mariottini |
| Web: http://ranger./ ~ gianluca |
| Email: gianluca@ |
| Office: |
| Computer Science and Engineering Dept. |
| University of Texas at Arlington |
| Nedderman Hall 309, 416 Yates Street |
| Arlington, TX 76019-0015 |
|
|
|
|
Downloads: [Syllabus] [Flyer]
1 - Course Overview
Vision is arguably the most important of the five senses, and is commonly used by humans
in many daily tasks (e.g., for recognizing objects or locations, as well as for safe navigation).
Endowing robots with such an expert and autonomous sense of vision has been a dream of
scientists and engineers for over half a century. Potential fields of interest are space
exploration, home-, industrial-, and medical-robotics.
In this course, students will be introduced to the basic techniques of autonomous vision-
based robot perception, recognition, localization and navigation. The topics covered include a
description of the main robotic and vision-sensing devices, as well as of the basic techniques
for image processing. Particular attention will be given to recognition and 3D scene and robot
estimation techniques that use single and multiple images. Emphasis will be given to
introducing the main up-to-date strategies for vision-based robot navigation (Position-, Image-
and Hybrid-Based Visual Servoing).
Throughout the course, students will work individually and in groups to analyze vision-based
robotics problems and to design software solutions. Matlab will be the primary programming
language/environment used in the assignments. After successfully completing this course,
students will be able to apply a variety of techniques for the design of efficient algorithms in
order to address complex problems of vision-based sensing, localization and navigation.
CSE, EE, MAE, BioE students are encouraged to register.
Course Prerequisites:
Office Hours:
The instructor is generally available before or after class and by appointment, as well as at the office hours scheduled above.
Since this is a special-topic course in robotics, and since each presented topic contains
many exciting sub-fields, the interested students who want to know more about a specific
problem are encouraged to schedule an appointment with Dr. Mariottini for additional
material.
2 - Course Material, Topics and Goals
- Basics on Robotics and Computer Vision
(TOPICS: Intro to Robotics; Pinhole camera model; Epipolar Geometry Toolbox)
- 08/30/10:
- Course Introduction: What is a robot? (Part1-pdf)
- 09/01/10:
- Robot Structures (Part2-pdf);
- Computer Vision: introduction, applications (Part3-pdf - until pg.14).
- 09/08/10:
- Simplified Camera Models (Part3-pdf - pg.14-26)
- 09/13/10:
- Introduction to EGT and 2-D stereo model (Part3-pdf - until the end) (MATLAB code).
- Feature Detection, Matching and Tracking
(TOPICS: Image noise; Linear Filtering; Non-linear filtering; Edge-Detection;
Corner Detection; Lucas-Kanade tracking; SIFT features)
- 09/13/10:
- Intro to Image Noise and Feature Extraction (edges, corners, etc.) (Part4-pdf - until pg.13) (MATLAB code);
- 09/15/10:
- Exercise 1 on pinhole and stereo camera models;
- Noise Models and Linear Filtering (Part4-pdf - until pg.19)
- 09/20/10:
- Non-linear Filtering, Edge Detection, Corner Detection (Part4-pdf - pg.19 until end)
- Kanade-Lucas Tracker (Part5-pdf - until pg.12);
- Exercise 2 on pinhole and distance calculation.
- 09/22/10:
- SIFT features: Keypoint and Descriptors extraction; Matching. (Part5-pdf - until end) (MATLAB code);
- Generalized Camera Models
(TOPICS: Rigid-body kinematics; Generalized camera-model)
- 09/27/10:
- Rigid body kinematics: ref. frames, rotations (Part6-pdf - until pg.10) (MATLAB code);
- 09/29/10:
- Rigid body kinematics: 3-D transformations and exercises (Part6-pdf - until pg.16);
- 10/03/10:
- Rigid body kinematics: generalized camera models (pinhole, stereo) (Part6-pdf - until end);
- Assignment No.1 is out (start: Oct.4th, 2010 at 5:20pm; deadline: Oct.13th, 2010 at 4:00pm!) (check Sect. 3 for more details).
- Vision-Based Estimation and Localization
(TOPICS: Camera Resectioning/Calibration; Least-Squares; Hough Transform; Camera Planar Resectioning; Stereo motion reconstruction; Multi-view Geometry)
- 10/06/10:
- Generalized stereo camera models (Part6-pdf - until end);
- Intro on Camera calibration/localization (Resectioning) (Part7-pdf);
- 10/11/10:
- Camera Localization: Resectioning and SVD (Part7-pdf);
- 10/13/10:
- Projects Presentation;
- Least-Squares Solution for Ax=b (Part8-pdf -until.pg.8)
- 10/18/10:
- Least-Squares Solution for Ax=b: Applications and RANSAC (Part8-pdf -until end) (MATLAB code);
- Assignment No.1: Correction of exercises.
- 10/20/10:
- Hough Transform (MATLAB code)
- Camera Localization from planar pattern (Part9-pdf -5) (MATLAB code);
- Assignment (no grade): see function Ex_CameraCalibrationIncomplete.m in the MATLAB code
- 10/25/10:
- Camera Localization from planar pattern (Part9-pdf -until end) (MATLAB code);
- 10/27/10:
- Introduction to Multiple-View Geometry and applications(Part10-pdf) (MATLAB code);
- 11/1/10:
- Multiple-view Geometry - Essential Matrix (Part10-pdf) (MATLAB code);
- 11/3/10:
- Multiple-view Geometry - Estimation and Scene reconstruction (Part10-pdf) (MATLAB code);
- Assignment No.2 is out (start:Nov.3rd, 2010 at 5:20pm; deadline: Nov.13th, 2010 at 4:00pm!) (the code of the solution is available here).
- 11/8/10:
- Multiple-view Geometry - Homography matrix (Part10-pdf) (MATLAB code);
- Vision-Based Robot Navigation
(TOPICS: Position-based visual Servoing; Trajectory generation and tracking; Image-Based Visual Servoing)
- 11/11/10:
- Introduction to Vision-based Robot Navigation (Part11-pdf);
- 11/15/10:
- Vision-based Robot Navigation: basic schemes.
- Vision-based object-tracking strategy (Part11-pdf);
- 11/17/10:
- Vision-Based Navigation for mobile robots: intro.
- Trajectory planning (Part12-pdf) (MATLAB code).
- 11/22/10:
- Trajectory tracking (Part12-pdf) (MATLAB code).
- Vision-Based Navigation for mobile robots: navigation using epipolar geometry (Part12-pdf);
- 11/24/10:
- Vision-Based Navigation for mobile robots: navigation using epipolar geometry.
- 11/29/10:
- Image-Based Visual Servoing: Introduction (Part13-pdf)
- 12/1/10:
- Image-Based Visual Servoing (Part13-pdf)
- 12/6/10: Project Presentations.
- 12/8/10: Project Presentations.
- 12/14/10: Project Presentations.
(please note that these course topics are preliminary and might undergo slight changes)
Matlab code
In order to run some of the code examples available in the course material, you need to download the Epipolar Geometry Toolbox (by G.L. Mariottini and D. Prattichizzo) for MATLAB. You can access here to EGT 2.0 (beta version).
Once you have downloaded it, include the EGT directory in your MATLAB path. At this point you should be ready to run the examples.
In order to run other MATLAB code you need to have the Image Processing Toolbox for MATLAB installed with your version.
Course text
There is no required textbook for this course.
Other course material [pdf of the lecture slides, technical papers, exercises,...] will be made
available on the course webpage (see above).
However, some suggested references which I've inspired to for the preparation of my course material are:
- B. Siciliano, L. Sciavicco, L. Villani, G. Oriolo
"Robotics: Modelling, Planning and Control", 3rd Edition, Springer, 2009
- E. Trucco, A. Verri
"Introductory Techniques for 3-D Computer Vision", Prentice Hall, 1998
- R. Hartley, A. Zisserman
"Multiple View Geometry", 2nd Edition, Cambridge Univ. Press 2004
- Y. Ma, S. Soatto, J. Kosecka, S. Sastry
"An Invitation to 3-D Vision. From Images to Geometric Models", Springer 2003
- R. Cipolla
"Computer Vision and Robotics", Cambridge University, UK (on-line course material)
As written above, most of the material used to design the course material has been taken from the mentioned sources which I acknowledge.
Course Goals
CSE 4392/5369 is designed to:
- Introduce the student to fundamentals of robotics and vision-based sensing.
- Explore the mathematical/algebraic foundations of single- and multi-view geometry for pose and 3-D scene estimation.
- Explore and familiarize with vision-based robot navigation techniques.
Course Outcomes
Upon successful completion of the course, each student will be able to:
- Recognize the features and peculiarities of different robotic and vision devices.
- Understand rigid body kinematics and relationships between 3-D reference frames.
- Describe image formation and camera models. Discuss/compare the characteristics of different vision sensing devices (pinhole, omnidirectional, stereo, etc.).
- Understand and discuss different strategies for feature detection, matching and tracking.
- Understand and implement 3D structure and motion estimation algorithms.
- Understand and design vision-based robot navigation algorithms.
3 - Course Assignments, Exam and Policies
Class attendance and participation
In CSE 4392/5369, the students will be presented with the state-of-the-art in vision-based robot control, localization and navigation technologies. Most of the topics are active fields of research in the robotics community. I strongly encourage the students to attend each class and to actively contribute with in-class discussion, when necessary. Students must arrive on time at class.
During the whole semester, I like to interact with the participants and ask them to actively participate to complete informal small in-class exercises. These informal activities will not be graded but will be used as a feedback or plan activities.
Homeworks and Course Project
Students will be graded based on homework assignments and a major course project.
Homeworks will mostly consist of theoretical questions, programming assignments in MATLAB, etc. related to the topics of the course.
To download the first assignment please login to WebCT with your UTA NetID. Once entered, you will be able to access to your specific section for this course (CSE 4392 or CSE 5369). Click on the Assignment you are interesting in accessing (e.g., Assignment No.1) and download the text of the exercise. Follow the instructions contained in there for the submission policy.
Dates for the homeworks will be announced in class. Regarding homeworks policies, please refer also to the Course Policy section.
The course project will focus on designing and implementing a particular algorithm from a list provided in class, and walking through details related to its analysis and design.
For their final project, students will have the possibility to use up to 4 iCreate robotic platforms (by iRobot) for the course project. USB cameras and laptops are also available to program the robot and process data from the sensors.
Additional projects related to the course topics and proposed by students can be added to the list. In evaluating the projects design, particular attention will be given to the projects that excel in creativity and effectiveness of their result. The course project can be done either individually or in a team.
Students will report their findings in a 8-10 page research report, and an in-class oral presentation (with Power Point slides) during last day of class.
For students enrolled in the graduate section CSE 5369 the homework assignments, as well as the course project, will contain additional problems which are not required for students of CSE 4392.
4 - Course/University Policies and Services
Tentatively, course grades will based on the following:
|
|
| Assignments | % of final grade |
| Homeworks (total) | 50 % |
| Course Project | 40 % |
| - Proj. Presentation | 10 % |
|
|
|
|
|
|
| Percent Grade | Letter |
| 90 %-100 % | A |
| 80 %-89.9 % | B |
| 70 %-79.9 % | C |
| 60 %-69.9 % | D |
| <60 % | F |
|
|
|
|
Final Project/Homework Late Submission Policy:
Late submissions for the final project will be penalized according to:
- Late by 1-24 hrs: 5% deducted from actual score.
- Late by 24-48 hrs: 15% deducted from actual score.
- Late by 48-72 hrs: 25% deducted from actual score.
- Late by more than 72 hrs: Will receive a zero.
Attendance and Participation
As stated above, attendance is strongly suggested at the first day and each class
session. Students are encouraged to arrive on time and attend the full class period.
Participants who need to miss class for religious observance or for a pressing
personal or family matter, should contact the instructor prior to missing class or as soon as
possible. Participants should plan on getting the information about the missed class from a
peer.
I strongly encourage in-class collegial behaviour as well as between the project group
members. NON collegial behavior includes working on other tasks during class time (text
messaging, e-mailing, Web surfing, doing crosswords/Sudoku, having private conversations,
etc.). Another example of non collegial behavior could be the creation of unconstructive
conflicts inside a group.
Finally, I positively value the students' active participation to in-class discussions. This is
extremely important because gives the instructor (and the students too!) a feedback on the
audience understanding.
Academic Honesty
All students are expected to pursue their academic careers with honesty and integrity.
"Scholastic dishonesty includes, but is not limited to, cheating, plagiarism, collusion, the
submission for credit of any work or materials that are attributable in whole or in part to
another person, taking an examination for another person, any act designed to give unfair
advantage to a student or the attempt to commit such acts" (Regents' Rules and
Regulations, Part One, Chapter VI, Section 3, Subsection 3.2, Subdivision 3.22.).
Students found guilty of dishonesty in their academic pursuits are subject to penalties that
may include suspension from the university. Any student found guilty of academic dishonesty
will receive a -100% for that work (homeworks, project, etc.) as well as having the
course grade lowered one full letter grade - in addition to any other penalties assessed
(suspension, expulsion, probation). These and other applying UTA rules, will be strictly
enforced. Any case of academic dishonesty will be treated in accordance with the UTA
Handbook of Operating Procedures or the Judicial Affairs website at
http://www2./discipline. If you do not understand this policy, it is your
responsibility to obtain clarification or any additional information you may require.
Students are allowed to discuss homework with classmates, but are not allowed to copy
the solutions of others or share solutions with others. All work turned in for grading must
be the student's own work.
Accommodations for Students With Disabilities
I will do my best to provide, on a flexible and individualized basis, reasonable
accommodations to students who have documented disability conditions (e.g., physical,
learning, psychiatric, vision, hearing, or systemic) that may affect their ability to participate in
course activities or to meet course requirements. If you require any accommodation based on
disability, please meet with the Instructor (with your supporting papers) in the privacy of his
office the first week of the semester to be sure you are appropriately accommodated.
Grievance Procedure
Anyone feeling that a dispute exists after the grading of any assignment or exam may
submit a written grievance. This grievance should identify the item in dispute and
arguments supporting the student's position. Grievances must be submitted in writing
within two class periods following the return of the assignment.
The instructor agrees to return a written response to the student's grievance within two
class periods from receipt of the grievance. If the error is due to wrongful calculation of points,
then no grievance needs to be submitted. If a written grievance is received, the instructor
reserves the right to re-grade the entire exam (not just the specific point in question).
Student Support Services Available
The University of Texas at Arlington supports a variety of student success programs to
help you connect with the University and achieve academic success. These programs
include learning assistance, developmental education, advising and mentoring, admission
and transition, and federally funded programs. Students requiring assistance academically,
personally, or socially should contact the Office of Student Success Programs at 817-272-6107
for more information and appropriate referrals.
Electronic Communication Policy
The University of Texas at Arlington has adopted the University "MavMail" address as
the sole official means of communication with students. MavMail is used to remind
students of important deadlines, advertise events and activities, and permit the University
to conduct official transactions exclusively by electronic means. For example, important
information concerning registration, financial aid, payment of bills, and graduation are
now sent to students through the MavMail system.
All students are assigned a MavMail account. Students are responsible for checking their
MavMail regularly. Information about activating and using MavMail is available at
http://www./oit/email/ .
File translated from
TEX
by
TTH,
version 3.85.
On 6 Dec 2010, 07:47.
|
