Human-Robot Collaboration Repo
Back to Top
Paper Repository on Human-Robot Collaboration
When referring to this site, please refer to its DOI address: https://doi.org/10.21229/M9PH2G.
Welcome to my paper repository! My name is
Hong (Herbert) Cai and I am a Ph.D. student at the University of California, Santa Barbara. I work with Prof. Yasamin Mostofi on the optimization of human-robot collaborations, with an emphasize on characterizing/predicting human task performance and response, and optimizing field sensing and navigation accordingly ( sample project page).
There has been great progress on different aspects of human-robot collaboration in recent years. This repository is then compiled to provide a comprehensive overview of the state-of-the-art on different aspects of human-robot collaboration. If you have any comments, feel free to email me at
hcai[at]ece.ucsb.edu. This repository will be updated regularly.
To find out more about our work on predicting human visual performance and its co-optimization with sensing and navigation, please check out our
research project page and RSS16 project page.
You can find and use our released data and machine learning pipeline for predicting human visual performance at
our RSS16 project page.
To find out more about me, check out
my personal page. To find out more about our research group and other exciting research work in our lab, check out Prof. Mostofi's page.
To refer to this paper repository, please use its DOI address:
Last updated August 2017
Overview of Human-Robot Collaboration
Thanks to the advances in areas such as perception, navigation, and robotic manipulation, robots are becoming more capable of accomplishing complicated tasks. There, however, still exist many tasks that robots cannot autonomously perform to a satisfactory level. A complex visual task, such as recognition and classification in the presence of uncertainty, is one example of such tasks. As such, robots can greatly benefit from human collaboration.
In this webpage, we have collected various research work related to human-robot collaborations. We group these papers into four major categories:
1) human modeling, 2) humans as supervisors, 3) humans and robots as peers, 4) robots as assistants, and 5) others. The first category includes papers that work on modeling different aspects of human factor, which is useful for human-robot collaborative designs. In each of the other subsequent three categories, humans and robots take different responsibilities in the operation. For instance, when humans are supervisors, they provide information, instructions, decisions, and/or certain services to the robots. When humans and robots are peers, they work together at the same level to achieve a common goal, such as object manipulation. When robots are assistants, they help humans with certain specific tasks while humans play a leading role in the operation. In addition, there are also research subjects that are not confined to any one of the three categories above, such as human-robot natural language dialog. These papers are listed in the category "Others".
The papers in each category (and sub-category) can be found via the corresponding hyper-link. In each category, the papers are listed in a chronological manner.
-- Human decision-making modeling
-- Human performance prediction and modeling
-- Human availability modeling
-- Ergonomics and general human modeling
Humans as supervisors
-- Decision-support systems
-- Querying human during operation
-- Learning from human during training
-- Human-in-the-loop control systems
-- Levels of autonomy/collaboration
-- Experimental studies
-- System designs
-- Review and survey papers
Humans and robots as peers
-- Collaborative manipulation
-- Collaborative planning
-- Human-robot decision fusion
-- Scalability of human-robot collaborations
-- Robot planning algorithms
-- Task allocation
-- Close-proximity collaborations and human safety
-- Human-robot musicianship
-- Experimental studies
-- System designs
-- Review and survey papers
Robots as assistants
-- Robot providing physical assistance
-- Robot providing information
-- How to provide assistance
-- Robot tutoring
-- Decision-support systems
-- Experimental studies
-- System designs
-- Human-robot dialog
-- Human-robot trust
These papers aim to model different aspects of human factors, such as decision-making dynamics, task performance, availability, and other ergonomic factors. Such modeling allows for the explicit consideration of human elements in the design of human-robot collaborations, which can lead to better and more efficient human-robot collaborations.
Human decision-making modeling
These papers study the human decision-making models/dynamics. They are commonly used to model human decision-making in control/robotics applications.
P. Reverdy, V. Srivastava, and N. Leonard, "
Modeling human decision making in generalized Gaussian multi-armed bandits," Proceedings of the IEEE 102.4 (2014): 544-571. M. McClelland and M. Campbell, "
Probabilistic modeling of anticipation in human controllers," IEEE Transactions on Systems, Man, and Cybernetics: Systems 43.4 (2013): 886-900. A. Stewart, M. Cao, A. Nedic, D. Tomlin, and N. Leonard, "
Towards human-robot teams: Model-based analysis of human decision making in two-alternative choice tasks with social feedback," Proceedings of the IEEE 100.3 (2012): 751-775. R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen, "
The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks," Psychological Review 113.4 (2006): 700.
Human performance prediction and modeling
These papers aim to predict or model human visual performance. In more recent ones, for instance, machine learning algorithms are developed to predict human performance based on several human studies. For instance, given an image, an algorithm learns to predict the probability of a person performing a visual task correctly. In other papers, elements of human visual performance are modeled, in a non-data-driven manner. For instance, some components in the imaging system or the human vision system are mathematically modeled.
H. Cai and Y. Mostofi, "
Asking for help with the right question by predicting human visual performance," Robotics: Science and Systems (RSS), 2016. R. T. Ionescu, B. Alexe, M. Leordeanu, M. Popescu, D. P. Papadopoulos, and V. Ferrari, "
How hard can it be? Estimating the difficulty of visual search in an image," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. M. Wakayama, D. Deguchi, K. Doman, I. Ide, H. Murase, and Y. Tamatsu, "
Estimation of the human performance for pedestrian detectability based on visual search and motion features," International Conference on Pattern Recognition (ICPR), 2012. D. Engel and C. Curio, "
Pedestrian detectability: Predicting human perception performance with machine vision," IEEE Intelligent Vehicles Symposium, 2011. S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, and S. Belongie, "
Visual recognition with humans in the loop," European Conference on Computer Vision (ECCV), 2010. R. Vollmerhausen, E. Jacobs, and R. Driggers, "
New metric for predicting target acquisition performance," Optical Engineering 43.11 (2004): 2806-2818. M. P. Eckstein, C. Abbey, and F. Bochud, "
A practical guide to model observers for visual detection in synthetic and natural noisy images," Handbook of Medical Imaging 1 (2000): 593-628. J. Johnson, "
Analysis of image forming systems," Selected Papers on Infrared Design, 1985.
Human availability modeling
These papers study the availability of humans for helping the robot. This modeling can be used in the optimization of how the robot should query the humans.
Ergonomics and general human modeling
These papers study various factors related to human working efficiency (e.g., fatigue, workload).
P. Varnell and F. Zhang, "
Characteristics of human pointing motions with acceleration," IEEE Conference on Decision and Control (CDC), 2015. J. R. Peters, V. Srivastava, G. S. Taylor, A. Surana, M. P. Eckstein, and F. Bullo, "
Human supervisory control of robotic teams: Integrating cognitive modeling with engineering design," IEEE Control Systems 35.6 (2015): 57-80. N. Ahmed, E. de Visser, T. Shaw, A. Mohamed-Ameen, M. Campbell, and R. Parasuraman, "
Statistical modelling of networked human-automation performance using working memory capacity," Ergonomics 57.3 (2014): 295-318. J. Y. C. Chen, and M. J. Barnes, "
Human-agent teaming for multirobot control: A review of human factors issues," IEEE Transactions on Human-Machine Systems 44.1 (2014): 13-29. A. Mao, E. Kamar, and E. Horvitz, "
Why stop now? Predicting worker engagement in online crowdsourcing," AAAI Conference on Human Computation and Crowdsourcing, 2013. Y. Boussemart and M. L. Cummings, "
Predictive models of human supervisory control behavioral patterns using hidden semi-Markov models," Engineering Applications of Artificial Intelligence 24.7 (2011): 1252-1262.
Humans as supervisors
In this category, we consider humans as the supervisors of the robotic operation. The humans may not be dedicated operators, but they provide information and/or service to the robots (e.g., commands, instructions, decisions based on the sensing data). Researchers have looked into different aspects in this category, including optimizing the decision-support to the human operator, optimizing robot's queries to the humans, and how robots can learn from human teaching.
These papers focus on developing decision-support systems in human-robot collaborations. Based on human modeling, these papers study how information should be provided to the human operator in an optimized manner. The optimization can include information gathering and presentation, and how to allocate/schedule information/tasks to the human operator.
K. Kalyanam, M. Pachter, M. Patzek, C. Rothwell, and S. Darbha, "
Optimal human-machine teaming for a sequential inspection operation," IEEE Transactions on Human-Machine Systems 46.4 (2016): 557-568. A. Vinod, T. Summers, and M. Oishi, "
User-interface design for MIMO LTI human-automation systems through sensor placement," American Control Conference (ACC), 2016. K. Savla, T. Temple, and E. Frazzoli, "
Human-in-the-loop vehicle routing policies for dynamic environments," IEEE Conference on Decision and Control (CDC), 2008. M. Cao, A. Stewart, and N. Leonard, "
Integrating human and robot decision-making dynamics with feedback: Models and convergence analysis," IEEE Conference on Decision and Control (CDC), 2008.
Attention allocation/Task scheduling:
J. R. Peters and L. F. Bertuccelli, "
Robust scheduling strategies for collaborative human-UAV missions," American Control Conference (ACC), 2016. C. Shannon, L. B. Johnson, K. F. Jackson, and J. P. How, "
Adaptive mission planning for coupled human-robot teams," American Control Conference (ACC), 2016. S. Hari, K. Sundar, S. Rathinam, and S. Darbha, "
Scheduling tasks for human operators in monitoring and surveillance applications," IFAC-PapersOnLine 49.32 (2016): 54-59. L. Jian, D. Yin, L. Shen, and J. Yang, "
Optimal attention allocation to visual search tasks of multi-UAVs based on operator model," IEEE International Conference on Mechatronics and Automation (ICMA), 2015. V. Srivastava, R. Carli, C. Langbort, and F. Bullo, "
Attention allocation for decision making queues," Automatica 50.2 (2014): 378-388. C. Murray and W. Park, "
Incorporating human factor considerations in unmanned aerial vehicle routing," IEEE Transactions on Systems, Man, and Cybernetics: Systems 43.4 (2013): 860-874. M. Majji and R. Rai, "
Autonomous task assignment of multiple operators for human robot interaction," American Control Conference (ACC), 2013. K. Savla and E. Frazzoli, "
A dynamical queue approach to intelligent task management for human operators," Proceedings of the IEEE 100.3 (2012): 672-686. J. Crandall, M. L. Cummings, M. Della Penna, and P. M. de Jong, "
Computing the effects of operator attention allocation in human control of multiple robots," IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 41.3 (2011): 385-397. L. F. Bertuccelli and M. L. Cummings, "
Scenario-based robust scheduling for collaborative human-UAV visual search tasks," IEEE Conference on Decision and Control and European Control Conference (CDC-ECC), 2011. A. Ortiz and C. Langbort, "
Scheduling multiple uninhabited aerial vehicles for target classification by single human operator," Journal of Aerospace Computing, Information, and Communication 8.12 (2011): 328-345.
Y. Diaz-Mercado, S. G. Lee, and M. Egerstedt, "
Human-swarm interactions via coverage of time-varying densities," Trends in Control and Decision-Making for Human-Robot Collaboration Systems, Springer International Publishing, 2017. A. Hocraffer and C. Nam, "
A meta-analysis of human-system interfaces in unmanned aerial vehicle (UAV) swarm management," Applied Ergonomics 58 (2017): 66-80. S. Fang, M. Peshkin, and M. MacIver, "
Human-in-the-loop active electrosense," Bioinspiration & Biomimetics 12.1 (2016). T. Gledhill, E. Mercer, and M. A. Goodrich, "
Modeling UASS for role fusion and human machine interface optimization," IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2013. J. Macbeth, M. L. Cummings, L. F. Bertuccelli, and A. Surana, "
Interface design for unmanned vehicle supervision through hybrid cognitive task analysis," Human Factors and Ergonomics Society Annual Meeting, 2012. M. Lewis, H. Wang, P. Velagapudi, P. Scerri, and K. Sycara, "
Using humans as sensors in robotic search," International Conference on Information Fusion, 2009. M. Oishi, I. Mitchell, A. M. Bayen, and C. J. Tomlin, "
Invariance-preserving abstractions of hybrid systems: Application to user interface design," IEEE Transactions on Control Systems Technology 16.2 (2008): 229-244. J. Cooper and M. A. Goodrich, "
Towards combining UAV and sensor operator roles in UAV-enabled visual search," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2008.
Querying human during operation
These papers focus on how robot/machine can best ask for human help in the operation phase. Some more recent papers below have also considered joint optimization frameworks where various elements in the robotic operation are jointly taken into account, e.g., sensing, communication, motion, and queries.
H. Cai and Y. Mostofi, "
When human visual performance is imperfect -- How to optimize the collaboration between one human operator and multiple field robots," Trends in Control and Decision-Making for Human-Robot Collaboration Systems, Springer International Publishing, 2017. H. Cai and Y. Mostofi, "
Asking for help with the right question by predicting human visual performance," Robotics: Science and Systems (RSS), 2016. H. Cai and Y. Mostofi, "
A human-robot collaborative traveling salesman problem: Robotic site inspection with human assistance," American Control Conference (ACC), 2016. O. Russakovsky, L. Li, and Li Fei-Fei, "
Best of both worlds: Human-machine collaboration for object annotation," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. H. Cai and Y. Mostofi, "
To ask or not to ask: A foundation for the optimization of human-robot collaborations," American Control Conference (ACC), 2015. S. Rosenthal, M. Veloso, and A. K. Dey, "
Is someone in this office available to help me?," Journal of Intelligent & Robotic Systems 66.1 (2012): 205-221. Y. Oren, A. Bechar, and Y. Edan, "
Performance analysis of a human-robot collaborative target recognition system," Robotica 30.05 (2012): 813-826. T. Kaupp, A. Makarenko, and H. Durrant-Whyte, "
Human-robot communication for collaborative decision making: A probabilistic approach," Robotics and Autonomous Systems 58.5 (2010): 444-456. S. Branson, C. Wah, F. Schroff, B. Babenko, P. Welinder, P. Perona, and S. Belongie, "
Visual recognition with humans in the loop," European Conference on Computer Vision (ECCV), 2010. S. Vijayanarasimhan and K. Grauman, "
What's it going to cost you? Predicting effort vs. informativeness for multi-label image annotations," IEEE Conference on Computer Vision and Pattern Recognition, 2009.
Learning from human during training
These papers study how robots can learn from human during the training phase. There are various research directions in this topic, including learning from demonstration, learning via crowdsourcing, reinforcement learning with human feedback, and active learning.
Learning from demonstrations:
H. C. Lin, T. Tang, Y. Fan, Y. Zhao, M. Tomizuka, and W. Chen, "
Robot learning from human demonstration with remote lead through teaching," European Control Conference (ECC), 2016. A. Mohseni-Kabir, C. Rich, S. Chernova, C. L. Sidner, and D. Miller, "
Interactive hierarchical task learning from a single demonstration," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2015. S. Alexandrova, M. Cakmak, K. Hsiao, and L. Takayama, "
Robot programming by demonstration with interactive action visualizations," Robotics: Science and Systems, 2014. B. Akgun, M. Cakmak, K. Jiang, and A. L. Thomaz, "
Keyframe-based learning from demonstration," International Journal of Social Robotics 4.4 (2012): 343-355. L. Cobo, C. Isbell Jr., and A. L. Thomaz, "
Automatic task decomposition and state abstraction from demonstration," International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2012. R. Toris, H. B. Suay, and S. Chernova, "
A practical comparison of three robot learning from demonstration algorithms," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2012. C. Mericli, M. Veloso, and H. Akin, "
Task refinement for autonomous robots using complementary corrective human feedback," International Journal of Advanced Robotic Systems 8.2 (2011): 16. B. D. Argall, B. Browning, and M. Veloso, "
Learning by demonstration with critique from a human teacher," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2007.
Learning via crowdsourcing:
A. Jain, D. Das, J. K. Gupta, and A. Saxena, "
Planit: A crowdsourcing approach for learning to plan paths from large scale preference feedback," IEEE International Conference on Robotics and Automation (ICRA), 2015. R. Toris, D. Kent, and S. Chernova, "
Unsupervised learning of multi-hypothesized pick-and-place task templates via crowdsourcing," IEEE International Conference on Robotics and Automation (ICRA), 2015. M. J. Y. Chung, M. Forbes, M. Cakmak, and R. P. Rao, "
Accelerating imitation learning through crowdsourcing," IEEE International Conference on Robotics and Automation (ICRA), 2014.
Reinforcement learning with human feedback:
W. B. Knox and P. Stone, " Framing reinforcement learning from human reward: Reward positivity, temporal discounting, episodicity, and performance," Artificial Intelligence 225 (2015): 24-50.
S. Griffith, K. Subramanian, J. Scholz, C. Isbell, and A. L. Thomaz, "
Policy shaping: Integrating human feedback with reinforcement learning," Advances in Neural Information Processing Systems (NIPS), 2013.
Robot active learning:
B. Hayes and B. Scassellati, "
Discovering task constraints through observation and active learning," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014. M. Cakmak and A. L. Thomaz, "
Designing robot learners that ask good questions," ACM/IEEE International Conference on Human-Robot Interaction, 2012. D. Silver, J. Bagnell, and A. Stentz, "
Active learning from demonstration for robust autonomous navigation," IEEE International Conference on Robotics and Automation (ICRA), 2012. S. Chernova and M. Veloso, "
Interactive policy learning through confidence-based autonomy," Journal of Artificial Intelligence Research 34.1 (2009): 1.
Human-in-the-loop control systems
These papers study how to incorporate human elements (e.g., control inputs) into control systems.
J. Jiang and A. Astolfi, "
Shared-control for a rear-wheel drive car: dynamic environments and disturbance rejection," IEEE Transactions on Human-Machine Systems, 2017. T. Hatanaka, N. Chopra, J. Yamauchi, and M. Fujita, "
A passivity-based approach to human-swarm collaboration and passivity analysis of human operators," Trends in Control and Decision-Making for Human-Robot Collaboration Systems, Springer International Publishing, 2017. K. Fitzsimons, E. Tzorakoleftherakis, and T. Murphey, "
Optimal human-in-the-loop interfaces based on Maxwell's demon," American Control Conference (ACC), 2016. R. Chipalkatty, H. Daepp, M. Egerstedt, and W. Book, "
Human-in-the-loop: MPC for shared control of a quadruped rescue robot," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2011. R. Chipalkatty and M. Egerstedt, "
Human-in-the-loop: Terminal constraint receding horizon control with human inputs," IEEE International Conference on Robotics and Automation (ICRA), 2010.
Levels of autonomy/collaboration
H. Saeidi, J. R. Wagner, and Y. Wang, "
A mixed-initiative haptic teleoperation strategy for mobile robotic systems based on bidirectional computational trust analysis," IEEE Transactions on Robotics, 2017. Y. Wang, "
Regret-based automated decision-making aids for domain search tasks using human-agent collaborative teams," IEEE Transactions on Control Systems Technology 24.5 (2016): 1680-1695. S. Ramchurn, J. Fischer, Y. Ikuno, F. Wu, J. Flann, and A. Waldock, "
A study of human-agent collaboration for multi-UAV task allocation in dynamic environments," International Joint Conference on Artificial Intelligence (IJCAI), 2015. P. Walker, S. Nunnally, M. Lewis, N. Chakraborty, and K. Sycara, "
Levels of automation for human influence of robot swarms," Human Factors and Ergonomics Society Annual Meeting, 2013. I. Tkach, A. Bechar, and Y. Edan, "
Switching between collaboration levels in a human-robot target recognition system," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 41.6 (2011): 955-967. M. L. Cummings and S. Bruni, "
Human-automated planner collaboration in complex resource allocation decision support systems," Intelligent Decision Technologies 4.2 (2010): 101-114. B. Hardin and M. A. Goodrich, "
On using mixed-initiative control: A perspective for managing large-scale robotic teams," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2009.
C. J. Shannon, D. C. Horney, K. F. Jackson, and J. P. How, "
Human-autonomy teaming using flexible human performance models: An initial pilot study," Advances in Human Factors in Robots and Unmanned Systems, 2017. M. L. Cummings, J. P. How, A. Whitten, and O. Toupet, "
The impact of human-automation collaboration in decentralized multiple unmanned vehicle control," Proceedings of the IEEE 100.3 (2012): 660-671.
G. L. Calhoun, M. A. Goodrich, J. R. Dougherty, and J. A. Adams, "
Human-autonomy collaboration and coordination toward multi-RPA missions," Remotely Piloted Aircraft Systems: A Human Systems Integration Perspective (2016): 109. C. Phillips-Grafflin, N. Alunni, H. B. Suay, J. Mainprice, D. Lofaro, D. Berenson, S. Chernova, R. W. Lindeman, and P. Oh, "
Toward a user-guided manipulation framework for high-DOF robots with limited communication," Intelligent Service Robotics 7.3 (2014): 121-131. D. Pitman and M. L. Cummings, "
Collaborative exploration with a micro aerial vehicle: A novel interaction method for controlling a MAV with a hand-held device," Advances in Human-Computer Interaction (2012): 18. T. Schouwenaars, M. J. Valenti, E. Feron, J. P. How, and E. Roche, "
Linear programming and language processing for human-unmanned aerial-vehicle team missions," Journal of Guidance, Control, and Dynamics 29.2 (2006): 303-313. T. Fong, C. Thorpe, and C. Baur, "
Collaboration, dialogue, human-robot interaction," Robotics Research, pp. 255-266, Springer Berlin Heidelberg, 2003.
Review and survey papers
M. A. Goodrich and M. L. Cummings, "
Human factors perspective on next generation unmanned aerial systems," Handbook of Unmanned Aerial Vehicles, 2015. B. D. Argall, S. Chernova, M. Veloso, and B. Browning, "
A survey of robot learning from demonstration," Robotics and Autonomous Systems 57.5 (2009): 469-483. M. L. Cummings and S. Bruni, "
Collaborative human-automation decision making," Springer Handbook of Automation, 2009. I. R. Nourbakhsh, K. Sycara, M. Koes, M. Yong, M. Lewis, and S. Burion, "
Human-robot teaming for search and rescue," IEEE Pervasive Computing 4.1 (2005): 72-79.
Humans and robots as peers
In this category, humans and robots are peers/teammates. Humans and robots work together on the same level and share similar responsibilities to finish the given tasks. There are several scenarios of such peer-to-peer human-robot collaborations, such as collaborative manipulation and collaborative planning. Researchers have studied various aspects of how to better design such collaborations, including robot planning algorithms, task allocation algorithms, cross-training of humans and robots, and guaranteeing human safety.
In collaborative manipulation, humans and robots work together on the same object/set of objects.
Predicting/Modeling human action:
D. Cehajic and S. Hirche, "
Estimating unknown object dynamics in human-robot manipulation tasks," IEEE International Conference on Robotics and Automation (ICRA), 2017. J. R. Medina, T. Lorenz, and S. Hirche, "
Considering human behavior uncertainty and disagreements in human-robot cooperative manipulation," Trends in Control and Decision-Making for Human-Robot Collaboration Systems, Springer International Publishing, 2017. H. Koppula and A. Saxena, "
Anticipating human activities using object affordances for reactive robotic response," IEEE Transactions on Pattern Analysis and Machine Intelligence 38.1 (2016): 14-29. C. Huang and B. Mutlu, "
Anticipatory robot control for efficient human-robot collaboration," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2016. P. Lasota and J. Shah, "
Analyzing the effects of human-aware motion planning on close-proximity human-robot collaboration," Human Factors: The Journal of the Human Factors and Ergonomics Society 57.1 (2015): 21-33. D. Kruse, R. J. Radke, and J. Wen, "
Collaborative human-robot manipulation of highly deformable materials," IEEE International Conference on Robotics and Automation (ICRA), 2015. K. P. Hawkins, N. Vo, S. Bansal, and A. F. Bobick, "
Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration," IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013. J. Mainprice and D. Berenson, "
Human-robot collaborative manipulation planning using early prediction of human motion," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013.
I. D. Walker, L. Mears, R. S. Mizanoor, R. Pak, S. Remy, and Y. Wang, "
Robot-human handovers based on trust," International Conference on Mathematics and Computers in Sciences and in Industry (MCSI), 2015. C. Huang, M. Cakmak, and B. Mutlu, "
Adaptive coordination strategies for human-robot handovers," Robotics: Science and Systems (RSS), 2015. H. Admoni, A. D. Dragan, S. S. Srinivasa, and B. Scassellati, "
Deliberate delays during robot-to-human handovers improve compliance with gaze communication," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2014. K. W. Strabala, M. K. Lee, A. D. Dragan, J. L. Forlizzi, S. S. Srinivasa, M. Cakmak, and V. Micelli, "
Towards seamless human-robot handovers," Journal of Human-Robot Interaction 2.1 (2013): 112-132.
In collaborative planning, humans and robots design the task plan together. For instance, the human may provide an initial task plan, based on which the robot generates an improved plan. Such collaboration allows drastic reduction of computational complexity of task planning, as compared to fully automatic planning.
J. Kim, C. Banks, and J. Shah, "
Collaborative planning with encoding of users' high-level strategies," AAAI Conference on Artificial Intelligence, 2017. T. Somers and G. Hollinger, "
Human-robot planning and learning for marine data collection," Autonomous Robots 40.7 (2016): 1123-1137. D. Yi, M. A. Goodrich, and K. Seppi, "
Homotopy-aware RRT*: Toward human-robot topological path-planning," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2016. A. S. Clare, "
Modeling real-time human-automation collaborative scheduling of unmanned vehicles," PhD Thesis, Massachusetts Institute of Technology, 2013. J. Rathje, L. Spence, and M. L. Cummings, "
Human-automation collaboration in occluded trajectory smoothing," IEEE Transactions on Human-Machine Systems 43.2 (2013): 137-148.
Human-robot decision fusion
K. G. Lore, N. Sweet, K. Kumar, N. Ahmed, and S. Sarkar, "
Deep value of information estimators for collaborative human-machine information gathering," ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS), 2016. N. Ahmed, E. Sample, and M. Campbell, "
Bayesian multicategorical soft data fusion for human-robot collaboration," IEEE Transactions on Robotics 29.1 (2013): 189-206. E. Kamar, S. Hacker, and E. Horvitz, "
Combining human and machine intelligence in large-scale crowdsourcing," International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2012. S. Ponda, N. Ahmed, B. Luders, E. Sample, T. Hoossainy, D. Shah, M. Campbell, and J. P. How, "
Decentralized information-rich planning and hybrid sensor fusion for uncertainty reduction in human-robot missions," AIAA Guidance, Navigation, and Control Conference, 2011. T. Kaupp, A. Makarenko, F. Ramos, B. Upcroft, S. Williams, and H. Durrant-Whyte, "
Adaptive human sensor model in sensor networks," International Conference on Information Fusion, 2005.
Scalability of human-robot collaborations
F. Bourgault, A. Chokshi, J. Wang, D. Shah, J. Schoenberg, R. Iyer, F. Cedano, and M. Campbell, "
Scalable Bayesian human-robot cooperation in mobile sensor networks," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2008. A. Makarenko, T. Kaupp, and H. Durrant-Whyte, "
Scalable human-robot interactions in active sensor networks," IEEE Pervasive Computing 2.4 (2003): 63-71.
Robot planning algorithms
In these work, researchers develop robot planning algorithms such that human factors are taken into account. For instance, the robot considers how its actions may affect its human co-workers when planning its actions.
X. Zhang, Y. Zhu, and H. Lin, "
Performance guaranteed human-robot collaboration through correct-by-design," American Control Conference (ACC), 2016. S. Pellegrinelli, H. Admoni, S. Javdani, and S. S. Srinivasa, "
Human-robot shared workspace collaboration via hindsight optimization," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016. A. D. Dragan and S. S. Srinivasa, "
Integrating human observer inferences into robot motion planning," Autonomous Robots 37.4 (2014): 351-368. M. Gielniak and A. L. Thomaz, "
Generating anticipation in robot motion," IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2011. G. Hoffman and C. Breazeal, "
Anticipatory perceptual simulation for human-robot joint practice: Theory and application study," AAAI Conference on Artificial Intelligence, 2008.
X. Wang, Z. Shi, F. Zhang, and Y. Wang, "
Dynamic real-time scheduling for human-agent collaboration systems based on mutual trust," Cyber-Physical Systems 1.2-4 (2015): 76-90. S. Rahman and Y. Wang, "
Dynamic affection-based motion control of a humanoid robot to collaborate with human in flexible assembly in manufacturing," ASME Dynamic Systems and Control Conference, 2015. M. Malvankar-Mehta and S. Mehta, "
Optimal task allocation in multi-human multi-robot interaction," Optimization Letters 9.8 (2015): 1787-1803.
Close-proximity collaborations and human safety
C. Young and F. Zhang, "
A learning algorithm to select consistent reactions to human movements," Trends in Control and Decision-Making for Human-Robot Collaboration Systems, Springer International Publishing, 2017. T. Iqbal, S. Rack, and L. D. Riek. "
Movement coordination in human-robot teams: a dynamical systems approach," IEEE Transactions on Robotics 32.4 (2016): 909-919. C. Liu and M. Tomizuka, "
Algorithmic safety measures for intelligent industrial co-robots," IEEE International Conference on Robotics and Automation (ICRA), 2016.
C. Breazeal, N. DePalma, J. Orkin, S. Chernova, and M. Jung, "
Crowdsourcing human-robot interaction: New methods and system evaluation in a public environment," Journal of Human-Robot Interaction 2.1 (2013): 82-111. J. Shah and C. Breazeal, "
An empirical analysis of team coordination behaviors and action planning with application to human-robot teaming," Human Factors 52.2 (2010): 234-245. J. L. Burke, R. R. Murphy, M. D. Coovert, and D. L. Riddle, "
Moonlight in Miami: Field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise," Human-Computer Interaction 19.1-2 (2004): 85-116.
D. Kruse, R. Radke, and J. Wen, "
Collaborative human-robot manipulation of highly deformable materials," IEEE International Conference on Robotics and Automation (ICRA), 2015. H. M. Do, C. Mouser, M. Liu, and W. Sheng, "
Human-robot collaboration in a mobile visual sensor network," IEEE International Conference on Robotics and Automation (ICRA), 2014. D. Lee, M. McClelland, J. Schneider, T. L. Yang, D. Gallagher, J. Wang, D. Shah, N. Ahmed, P. Moran, B. Jones, and T. S. Leung, "
Distributed, collaborative human-robotic networks for outdoor experiments in search, identify and track," Proceedings of SPIE, 2010. M. B. Dias, B. Kannan, B. Browning, E. G. Jones, B. Argall, M. F. Dias, M. Zinck, M. M. Veloso, and A. J. Stentz, "
Sliding autonomy for peer-to-peer human-robot teams," Intelligent Conference on Intelligent Autonomous Systems (IAS), 2008. C. Breazeal, A. Brooks, J. Gray, G. Hoffman, C. Kidd, H. Lee, J. Lieberman, A. Lockerd, and D. Mulanda, "
Humanoid robots as cooperative partners for people," International Journal of Humanoid Robots 1.2 (2004): 1-34. G. Hoffman and C. Breazeal, "
Collaboration in human-robot teams," AIAA Intelligent Systems Technical Conference, 2004.
Review and survey papers
J. Shah, J. Saleh, and J. Hoffman, "
Review and synthesis of considerations in architecting heterogeneous teams of humans and robots for optimal space exploration," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37.5 (2007): 779-793. R. Murphy, "
Human-robot interaction in rescue robotics," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 34.2 (2004): 138-153.
Robots as assistants
In this category, humans play major roles in the collaborative operations and robots assist the humans by providing information/services. For instance, in an industrial manufacturing setting, humans work on assembling parts while robots move parts from one place to another.
Robot providing physical assistance
V. Unhelkar, H. Siu, and J. Shah, "
Comparative performance of human and mobile robotic assistants in collaborative fetch-and-deliver tasks," ACM/IEEE International Conference on Human-robot Interaction (HRI), 2014. T. Carlson and Y. Demiris, "
Collaborative control for a robotic wheelchair: Evaluation of performance, attention, and workload," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 42.3 (2012): 876-888.
Robot providing information
H.-C. Wang, R. K. Katzschmann, B. Araki, S. Teng, L. Giarre, and D. Rus, "
Enabling independent navigation for visually impaired people through a wearable vision-based feedback system," IEEE International Conference on Robotics and Automation (ICRA), 2017. M. J. Y. Chung, A. Pronobis, M. Cakmak, D. Fox, and R. P. Rao, "
Autonomous question answering with mobile robots in human-populated environments," IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.
C. Wu, J. Zhang, B. Selman, S. Savarese, and A. Saxena, "
Watch-Bot: Unsupervised learning for reminding humans of forgotten actions," IEEE International Conference on Robotics and Automation (ICRA), 2016.
How to provide assistance
T. Munzer, Y. Mollard, and M. Lopes, "
Impact of robot initiative on human-robot collaboration," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2017.
J. Baraglia, M. Cakmak, Y. Nagai, R. P. Rao, and M. Asada, "
Efficient human-robot collaboration: When should a robot take initiative?," The International Journal of Robotics Research, 2017.
O. Zuckerman, G. Hoffman, D. Kopelman-Rubin, A. B. Klomek, N. Shitrit, Y. Amsalem, and Y. Shlomi, "
KIP3: Robotic companion as an external cue to students with ADHD," International Conference on Tangible, Embedded, and Embodied Interaction, 2016. G. Gordon, S. Spaulding, J. K. Westlund, J. J. Lee, L. Plummer, M. Martinez, M. B. Das, and C. Breazeal, "
Affective personalization of a social robot tutor for children's second language skills," AAAI Conference on Artificial Intelligence, 2016.
These papers study aspects of human-robot collaborations that are not confined by any one of the three categories above, e.g., human-robot natural language dialog.
D. Arumugam, S. Karamcheti, N. Gopalan, L. Wong, and S. Tellex, "
Accurately and efficiently interpreting human-robot instructions of varying granularities," Robotics: Science and Systems (RSS), 2017. R. Scalise, S. Rosenthal, and S. S. Srinivasa, "
Natural language explanations in human-collaborative systems," ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2017. J. Thomason, S. Zhang, R. J. Mooney, and P. Stone, "
Learning to interpret natural language commands through human-robot dialog," International Joint Conference on Artificial Intelligence (IJCAI), 2015. S. Tellex, R. A. Knepper, A. Li, N. Roy, and D. Rus, "
Asking for help using inverse semantics," Robotics: Science and Systems (RSS), 2014. R. Deits, S. Tellex, P. Thaker, D. Simeonov, T. Kollar, and N. Roy, "
Clarifying commands with information-theoretic human-robot dialog," Journal of Human-Robot Interaction 2.2 (2013): 58-79. S. Rosenthal, M. Veloso, and A. K. Dey, "
Acquiring accurate human responses to robots' questions," International Journal of Social Robotics 4.2 (2012): 117-129.
B. Sadrfaridpour, H. Saeidi, J. Burke, K. Madathil, and Y. Wang, "
Modeling and control of trust in human-robot collaborative manufacturing," Robust Intelligence and Trust in Autonomous Systems, 2016. A. Clare, M. L. Cummings, and N. Repenning, "
Influencing trust for human-automation collaborative scheduling of multiple unmanned vehicles," Human Factors: The Journal of the Human Factors and Ergonomics Society 57.7 (2015): 1208-1218.