Human-Robot Collaboration Repo Back to Top

Paper Repository on Human-Robot Collaboration
When referring to this site, please refer to its DOI address: https://doi.org/10.21229/M9PH2G.

Welcome to my paper repository! My name is Hong (Herbert) Cai and I am a Ph.D. student at the University of California, Santa Barbara. I work with Prof. Yasamin Mostofi on the optimization of human-robot collaborations, with an emphasize on characterizing/predicting human task performance and response, and optimizing field sensing and navigation accordingly (sample project page).

There has been great progress on different aspects of human-robot collaboration in recent years. This repository is then compiled to provide a comprehensive overview of the state-of-the-art on different aspects of human-robot collaboration. If you have any comments, feel free to email me at hcai[at]ece.ucsb.edu. This repository will be updated regularly.

To find out more about our work on predicting human visual performance and its co-optimization with sensing and navigation, please check out our research project page and RSS16 project page.

You can find and use our released data and machine learning pipeline for predicting human visual performance at our RSS16 project page.

To find out more about me, check out my personal page. To find out more about our research group and other exciting research work in our lab, check out Prof. Mostofi's page.

To refer to this paper repository, please use its DOI address: https://doi.org/10.21229/M9PH2G.

Last updated December 2018

Overview of Human-Robot Collaboration

Thanks to the advances in areas such as perception, navigation, and robotic manipulation, robots are becoming more capable of accomplishing complicated tasks. There, however, still exist many tasks that robots cannot autonomously perform to a satisfactory level. A complex visual task, such as recognition and classification in the presence of uncertainty, is one example of such tasks. As such, robots can greatly benefit from human collaboration.

In this webpage, we have collected various research work related to human-robot collaborations. We group these papers into four major categories: 1) human modeling, 2) humans as supervisors, 3) humans and robots as peers, 4) robots as assistants, and 5) others. The first category includes papers that work on modeling different aspects of human factor, which is useful for human-robot collaborative designs. In each of the other subsequent three categories, humans and robots take different responsibilities in the operation. For instance, when humans are supervisors, they provide information, instructions, decisions, and/or certain services to the robots. When humans and robots are peers, they work together at the same level to achieve a common goal, such as object manipulation. When robots are assistants, they help humans with certain specific tasks while humans play a leading role in the operation. In addition, there are also research subjects that are not confined to any one of the three categories above, such as human-robot natural language dialog. These papers are listed in the category "Others".

The papers in each category (and sub-category) can be found via the corresponding hyper-link. In each category, the papers are listed in a chronological manner.

Human modeling
-- Human decision-making modeling
-- Human performance prediction and modeling
-- Human availability modeling
-- Ergonomics and general human modeling

Humans as supervisors
-- Decision-support systems
-- Querying human during operation
-- Learning from human during training
-- Human-in-the-loop control systems
-- Levels of autonomy/collaboration
-- Experimental studies
-- System designs
-- Review and survey papers

Humans and robots as peers
-- Collaborative manipulation
-- Collaborative planning
-- Human-robot decision fusion
-- Scalability of human-robot collaborations
-- Robot planning algorithms
-- Task allocation
-- Cross-training
-- Close-proximity collaborations and human safety
-- Human-robot musicianship
-- Experimental studies
-- System designs
-- Review and survey papers

Robots as assistants
-- Robot providing physical assistance
-- Robot providing information
-- How to provide assistance
-- Robot tutoring
-- Decision-support systems
-- Experimental studies
-- System designs

Others
-- Human-robot dialog
-- Human-robot trust

Human modeling

These papers aim to model different aspects of human factors, such as decision-making dynamics, task performance, availability, and other ergonomic factors. Such modeling allows for the explicit consideration of human elements in the design of human-robot collaborations, which can lead to better and more efficient human-robot collaborations.

Human decision-making modeling
These papers study the human decision-making models/dynamics. They are commonly used to model human decision-making in control/robotics applications.

Human performance prediction and modeling
These papers aim to predict or model human visual performance. In more recent ones, for instance, machine learning algorithms are developed to predict human performance based on several human studies. For instance, given an image, an algorithm learns to predict the probability of a person performing a visual task correctly. In other papers, elements of human visual performance are modeled, in a non-data-driven manner. For instance, some components in the imaging system or the human vision system are mathematically modeled.

Human availability modeling
These papers study the availability of humans for helping the robot. This modeling can be used in the optimization of how the robot should query the humans.

Ergonomics and general human modeling
These papers study various factors related to human working efficiency (e.g., fatigue, workload).

Humans as supervisors

In this category, we consider humans as the supervisors of the robotic operation. The humans may not be dedicated operators, but they provide information and/or service to the robots (e.g., commands, instructions, decisions based on the sensing data). Researchers have looked into different aspects in this category, including optimizing the decision-support to the human operator, optimizing robot's queries to the humans, and how robots can learn from human teaching.

Decision-support systems
These papers focus on developing decision-support systems in human-robot collaborations. Based on human modeling, these papers study how information should be provided to the human operator in an optimized manner. The optimization can include information gathering and presentation, and how to allocate/schedule information/tasks to the human operator.

Information collection/presentation:

Attention allocation/Task scheduling:

Interface design:

Querying human during operation
These papers focus on how robot/machine can best ask for human help in the operation phase. Some more recent papers below have also considered joint optimization frameworks where various elements in the robotic operation are jointly taken into account, e.g., sensing, communication, motion, and queries.

Learning from human during training
These papers study how robots can learn from human during the training phase. There are various research directions in this topic, including learning from demonstration, learning via crowdsourcing, reinforcement learning with human feedback, and active learning.

Learning from demonstrations:

Learning via crowdsourcing:

Reinforcement learning with human feedback:

Robot active learning:

Interactive learning:

Human-in-the-loop control systems
These papers study how to incorporate human elements (e.g., control inputs) into control systems.

Levels of autonomy/collaboration

Experimental studies

System designs

Review and survey papers

Humans and robots as peers

In this category, humans and robots are peers/teammates. Humans and robots work together on the same level and share similar responsibilities to finish the given tasks. There are several scenarios of such peer-to-peer human-robot collaborations, such as collaborative manipulation and collaborative planning. Researchers have studied various aspects of how to better design such collaborations, including robot planning algorithms, task allocation algorithms, cross-training of humans and robots, and guaranteeing human safety.

Collaborative manipulation
In collaborative manipulation, humans and robots work together on the same object/set of objects.

Predicting/Modeling human action:

Object handover:

Collaborative planning
In collaborative planning, humans and robots design the task plan together. For instance, the human may provide an initial task plan, based on which the robot generates an improved plan. Such collaboration allows drastic reduction of computational complexity of task planning, as compared to fully automatic planning.

Human-robot decision fusion

Human-robot adaptation

Scalability of human-robot collaborations

Robot planning algorithms
In these work, researchers develop robot planning algorithms such that human factors are taken into account. For instance, the robot considers how its actions may affect its human co-workers when planning its actions.

Task allocation

Cross-training

Close-proximity collaborations and human safety

Human-robot musicianship

Experimental studies

System designs

Review and survey papers

Robots as assistants

In this category, humans play major roles in the collaborative operations and robots assist the humans by providing information/services. For instance, in an industrial manufacturing setting, humans work on assembling parts while robots move parts from one place to another.

Robot providing physical assistance

Robot providing information

How to provide assistance

Robot tutoring

Decision-support systems

Experimental studies

System designs

Others

These papers study aspects of human-robot collaborations that are not confined by any one of the three categories above, e.g., human-robot natural language dialog.

Human-robot dialog

Human-robot trust