Brown CS 2951-P

Introduction

CS 2951-P is a seminar course on Human-Robot Interaction (HRI). The theme of the HRI Seminar in Fall 2014 is "Building Human-usable 3D Maps for Robots". Consider how interactive 2D maps of the world have become commonplace, almost essential, in our daily lives. Map services from companies such as Mapquest, Google, Apple, etc. are used in many different ways to help our decision making about where to go, where to shop, where to live, and a plethora of our activities. Maps from these services contain not only physical descriptions of space (such as geospatial locations, metric geographic terrain, street views) but also important semantic information (navigation graphs, directory labels, ratings). Such maps are also the basis for autonomous decision making for advising people as well as for robot control. For example, route directions provided by a mapping service between locations is generated by a planner that searches for optimal paths in the map. Similarly, a robot will use a planner on a map to autonomously move across the locations in a building. However, this requires a map building phase followed by a labeling phase and then a planning phase. Building on advances in depth sensing technologies, we will focus on building maps using Red-Green-Blue-Depth (RGBD) sensors, such as the Microsoft Kinect, followed by interactively labeling these maps so that they can be used to plan and execute a task by robots such as the Willow Garage PR2.

This course focuses on SLAM part of robotics and how human interaction on these maps would help in building better maps. We will cover topics that address the perceptual challenges involved in robotics for estimating 3D maps. The course will go over basic concepts in robot sensing (from odometry, laser rangefinding, RGBD sensors and cameras) and robot mapping and state estimation (including Simultaneous Localization and Mapping, or SLAM). We will additionally cover new exploratory ideas for semantic labeling of maps through recent publications in robot supervisory control, robot learning from demonstration, semantic mapping, geometry sketching, etc. We will essentially discuss research papers covering the following areas:

• Smoothing and downsampling RGBD data,
• Basic RGBD Mapping,
• Current methods in SLAM (e.g., SGD, iSAM, FastSLAM, EKF, ICP)
• Human-in-the-loop map building, labeling, and supervisory control .

Course Staff

Instructor

Prof. Chad Jenkins
CIT 327
Office Hours: M 1:30-2:30pm, Tu 1:30-3pm

Teaching Assistant

Karthik Desingh
CIT 315
Office Hours: Fr 10am-12pm

Admin

Prof. Jenkins administrative support person is Suzanne Alden.

Meeting time/place

Monday 3:00-5:20 (M Hour)
CIT Lubrano (Class session) and CIT 115 (Lab session)

Mailing list:

cs2951p-hri-seminar@googlegroups.com

Blogpost discussion:

http://cs2951-hri-seminar.blogspot.com/

Optional Textbooks

Multi-view Geometry in Computer Vision, Hartley and Zisserman, Cambridge University Press, 2004.

Telerobotics, automation, and human supervisory control, Sheridan, MIT Press, 1992.

Prerequisites:

Linear Algebra (CS 53 or equivalent)

and one of:
Autonomous Robotics (CS 148), Computer Vision (CS 143), Computer Graphics (CS 123), Artificial Intelligence (CS 141)

or instructor permission

Schedule (tentative)

Week	Date	Class session	Presenters	Lab session	Slides	Projects
1	Sept 8	Initialization Introduction, Paper assignments, etc.	Instructor	RGBD SLAM demos	Slides	Prelim Projects 1, 2, 3 - Out
2	Sept 15	Prelim Project Primer In-class refresher: 3D transforms and odometry Fast Iterative Alignment of Pose Graphs with Poor Initial Estimates, Olson et al., International Conference on Robotics and Automation, 2006. Point Set Surfaces, Alexa et al., IEEE Visualization, 2001.	Instructor Alexis Carl	Introduction to Point Cloud Library	Refresher Slides MLS SGD-SLAM
3	Sept 22	Hacking session on Prelim Projects		Hacking session on Prelim Projects
4	Sept 29	ICP-based Mapping RGBD Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments, Henry et al., International Symposium on Experimental Robotics, 2010. 3D Mapping with an RGB-D Camera Endres et al., IEEE Transactions on Robotics, 30(1), 2014.	Karthik Matt	Hacking session on Prelim Projects	RGBD-Mapping_Henry 3D Mapping_Endres
5	Oct 6	Keyframe-based Mapping On unifying key-frame and voxel-based dense visual SLAM at large scales, Meilland and Comport, IEEE Intelligent Robots and Systems, 2013. LSD-SLAM: Large-Scale Direct Monocular SLAM, Engel et al., European Conference on Computer Vision (ECCV), 2014.	Jun Ki Lee Onur	Hacking session on Prelim Projects	Unifying Key-frame and Voxel based SLAM LSD SLAM
6	Oct 13	Fall Weekend Holiday!				Prelim Projects due; Project 4 - SLAM out
7	Oct 20	Mapping by Factor Graph Optimization iSAM: Incremental Smoothing and Mapping, Kaess et al., IEEE Transactions on Robotics, 24(6) 2008. Inference on networks of mixtures for robust robot mapping, Olson and Agarwal, Robotics: Science and Systems, 2012.	Lingzhu Zhiqiang	Hacking session on SLAM project	iSAM Max-mixtures
8	Oct 27	3D Reconstruction KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera Izadi et al, ACM UIST, 2011. PTAM, George Klein et al., ISMAR, 2007.	Carl Karthik	Hacking session on SLAM project	KinectFusion PTAM
9	Nov 3	Geometry by Annotation 3-Sweep: Extracting Editable Objects from a Single Photo, Chen et al., ACM SIGGRAPH Asia, 2013. VideoTrace: Rapid interactive scene modelling from video, van den Hengel et al., ACM SIGGRAPH, 2007. KML - Keyhole Markup Language tutorial and hands-on exercise.	Karthik Alexis	Hacking session on SLAM project	VideoTrace
10	Nov 10	Filter based SLAM FastSLAM, Michael Montemerlo et al, AAAI 2002. Grasping and Affordances Strategies for Human-in-the-Loop Robotic Grasping, Hsiao et., HRI, 2012. Localizing Handle-like Grasp Affordances in 3D Point Clouds, Platt et al., ISER 2014 NASA Valkyrie Affordance Templates. Paper draft available as hardcopy through course staff.	Joey - FastSLAM		FastSLAM
11	Nov 17	Hacking session on final project		Hacking session on final project
12	Nov 24	Thanksgiving Break!		Hacking session on final project
13	Dec 1	Strategies for Human-in-the-Loop Robotic Grasping, Hsiao et., HRI, 2012.	Joey	Hacking session on final project	Strategies in Grasping
14	Dec 8	Final Project demos and Presentation				Final Project due.

Projects and Grading

Full participation in this course consists of presenting 2 papers in class, participating in paper discussions, and completing 5 projects. The 5 projects are categorized as:

• 3 prelminary projects (to build a basic mapping system),
• a SLAM project (to build a working SLAM system) and
• a final project (that enables human-robot interaction through a built map).

CS 2951-P projects are graded as “checked” (completed) or “not checked” (incomplete). Please create a repository for your work (preferably on github) such that it can be retrieved by the course staff. All the project deadlines will have a demo by the students in the lab session. Final project will also require a presentation regarding the various methods used and results. The final project involves the design and implementation of a human-usable interface for labeling a map built from the course programming assignments.

Paper discussions and submission of paper summaries will occur through the course discussion forum. For full credit, the summary of a given paper must be entered into the discussion forum before the presentation of this paper in class.

To receive an A grade in the course, you must:

• select and present 2 papers in class
• write a summary of each paper presented in class
• complete and have checked all 5 projects checked by the end of reading period.

To receive an B grade in the course, you must:

• select and present 2 papers in class
• write a summary for 50% of the papers presented in class
• complete 2 of the 3 preliminary projects
• complete 1 of the SLAM or final projects

Git repositories will be used for project implementation, version control, and submission. Project implementations are submitted as branches in your assigned repository, named "projectX" for X project. These branches must be submitted prior to the due date for each assignment. Your implementation will be checked out and executed by the course staff. You will be notified by the course staff whether your implementation is sufficient for checking off the assignment. If your assignment is insufficient for receiving a check, your group is allowed one regrade (per assignment) with 2 weeks of notification. If deemed necessary, the course staff may require an interactive demonstration of your implementation and/or a web-based written report.

Final Grading

All grading will be finalized on December 10, 2014, after final project presentations.

Repositories

The course staff will provide a git repository (through github) for each student group to check in project source code and reports, unless you request using separate service or repository. Please refer to this tutorial for an in-depth introduction to git and version control. Although it has a coarse language choice, this is also a great and accessible tutorial. We expect students to use these repositories for collaborative development as well as project submission. It is the responsibility of each student group to ensure their repository adheres to the Collaboration Policy and submission standards for each assignment. Submission standards and examples will be described for each assignment.

Late Policy

Do not be late submitting assignments. The course staff reserves the right to not grade late submissions.

Collaboration Policy

This policy covers all course material and assignments unless otherwise stated. Course material information and documentation may be discussed with anyone. Course material does not include assignment handouts. Assignments may be discussed with the other CS 2951-P students. Discussions may make use of a whiteboard or paper. Discussions outside of your group cannot include writing or debugging code on a computer. You may take notes away from these discussions, provided these notes do not include any source code. The code for your implementation may not be shown to anyone outside of your group, including granting access to repositories or careless lack of protection. You do not need to hide the screen from anyone, but you should not attempt to show anyone your code. When you are done using any robot device such that another group may use it, you must remove all code you have put onto the device. You may not share your code with others outside of your group. At any time, you may show others the implemented program running on a device or simulator, but you may not discuss specific debugging details about your code while doing so. Should you fail to abide by this policy, you will receive no credit for this course. Brown also reserves the right to pursue any means necessary to ensure compliance. This includes, but is not limited to prosecution through Brown University’s Student Conduct Procedures, which can result in your suspension or expulsion from the University