Readable Grant Applications

Emoters, Inc. Application for a 2016 NSF SBIR grant. Design, deployment, and algorithmic optimization of zoomorphic, interactive robot companions. PI: W. Bradley Knox. Awarded December 2016.
pdf

Journal Articles

W. Bradley Knox*, Stephane Hatgis-Kessell*, Serena Booth, Scott Niekum, Peter Stone, and Alessandro Allievi. Models of human preference for learning reward functions. TMLR 2024.
Spotlight paper at The 5th Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2022)
*Equal contribution
paper page
pdf
arxiv page

W. Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone. Reward (Mis)design for Autonomous Driving. Artificial Intelligence (AIJ), 316, January 2023.
paper page
pdf
AIJ page (open access)
teaser video
Originally published on arxiv in 2021.

Guangliang Li, Shimon Whiteson, W. Bradley Knox, and Hayley Hung. Social interaction for efficient agent learning from human reward. Autonomous Agents and Multi-Agent Systems (JAAMAS). 32 (1), 1-25. January 2018. (Published online July 2017)
pdf
JAAMAS page (open access)

W. Bradley Knox and Peter Stone. Framing Reinforcement Learning from Human Reward: Reward Positivity, Temporal Discounting, Episodicity, and Performance. Artificial Intelligence, 225, 24–50. August 2015.
pdf
Artificial Intelligence Journal page

Guangliang Li, Shimon Whiteson, W. Bradley Knox, and Hayley Hung.  Using informative behavior to increase engagement while learning from human reward. Autonomous Agents and Multi-Agent Systems (JAAMAS). 30 (5), 826–848. September 2016. (Published online August 2015)
pdf
JAAMAS page (open access)

Saleema Amershi*, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35 (4): 105-120. December 2014.
pdf
AI Magazine page
*All authors contributed equally

A. Ross Otto, W. Bradley Knox, Art Markman, and  Bradley C. Love. Behavioral and Physiological Signatures of Reflective Exploratory Choice. Cognitive, Affective, & Behavioral Neuroscience. 1-17. March 2014
pdf
CABN Journal page

Jin Joo Lee, W. Bradley Knox, Jolie Baumann, Cynthia Breazeal, and David DeSteno. Computationally Modeling Interpersonal Trust. Frontiers in Psychology, 4, 893. 2013.
pdf
Frontiers in Psychology article page

W. Bradley Knox, Brian D. Glass, Bradley C. Love, W. Todd Maddox, and Peter Stone. How Humans Teach Agents: A New Experimental Perspective. International Journal of Social Robotics. 4(4) , 409-421. July 2012.
pdf
The final publication is available here at www.springerlink.com.

W. Bradley Knox*, A. Ross Otto, Peter Stone, and Bradley C. Love. The Nature of Belief-Directed Exploratory Choice in Human Decision-making. Frontiers in Psychology, 2, 398. 2012.
pdf
Frontiers in Psychology article page
*First two authors contributed equally

Juhyun Lee, W. Bradley Knox, and Peter Stone. Inter-Classifier Feedback for Human-Robot Interaction in a Domestic Setting. Journal of Physical Agents, 2(2):41–50, July 2008. Special Issue on Human Interaction with Domestic Robots
pdf

Conference Papers

Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, W. Bradley Knox, Chelsea Finn, Scott Niekum. Scaling laws for reward model overoptimization in direct alignment algorithms. NeurIPS 2024.
pdf
arxiv

Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W. Bradley Knox, Dorsa Sadigh. Contrastive Preference Learning: Learning from Human Feedback without RL. ICLR 2024. ICLR 2024.
pdf
arxiv

W. Bradley Knox, Stephane Hatgis-Kessell, Sigurdur Orn Adalgeirsson, Serena Booth, Anca Dragan, Peter Stone, and Scott Niekum. Learning Optimal Advantage from Preferences and Mistaking it for Reward. arXiv:2310.02456 [cs.LG]. Oct 2023. AAAI 2024.
pdf
arxiv

Serena Booth, W. Bradley Knox, Julie Shah, Scott Niekum, Peter Stone, and Alessandro Allievi. The Perils of Trial-and-Error Reward Design: Misdesign through Overfitting and Invalid Task Specifications. In Proceedings of the Thirty-Seventh National Conference on Artificial Intelligence (AAAI), February, 2023.
Selected for Oral Presentation
pdf

Cassidy Curtis, Sigurdur Orn Adalgeirsson, Horia Stefan Ciurdar, Peter McDermott, JD Velásquez, W. Bradley Knox, Alonso Martinez, Dei Gaztelumendi, Norberto Adrian Goussies, Tianyu Liu, and Palash Nandy. Toward Believable Acting for Autonomous Animated Characters. In Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG). November, 2022. 
Best Presentation Award (presented by Cassidy Curtis)
pdf
ACM page with video (open access)
MIG 2022
paper website

Yuchen Cui*, Qiping Zhang*, Alessandro Allievi, Peter Stone, Scott Niekum, and W. Bradley Knox. The EMPATHIC Framework for Task Learning from Implicit Human Feedback. In Proceedings of the Conference on Robot Learning (CoRL). November, 2020.
pdf
arXiv
paper website (with talks and links to code)
* Equal contribution

W. Bradley Knox, Samuel Spaulding, and Cynthia Breazeal. Learning from the Wizard: Programming Social Interaction through Teleoperated Demonstrations. In Proceedings of the Fifteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). May, 2016.
pdf
AAMAS 2016

Guangliang Li, Hayley Hung, Shimon Whiteson, and W. Bradley Knox.  Learning from Human Reward Benefits from Socio-competitive Feedback. In Proceedings of the IEEE International Conference on Development and Learning and on Epigenetic Robots (ICDL-EpiRob). October, 2014.
pdf
ICDL-EpiRob

W. Bradley Knox, Cynthia Breazeal, and Peter Stone. Training a Robot via Human Feedback: A Case Study. In Proceedings of the International Conference on Social Robotics (ICSR). October, 2013.
Best Paper Award
pdf
talk
ICSR 2013

Guangliang Li, Hayley Hung, Shimon Whiteson, and W. Bradley Knox. Using Informative Behavior to Increase Engagement in the TAMER Framework. In Proceedings of the Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). May, 2013.
pdf
AAMAS 2013

W. Bradley Knox and Peter Stone. Learning Non-Myopically from Human-Generated Reward. In Proceedings of the International Conference on Intelligent User Interfaces (IUI). March 2013.
pdf
IUI 2013

W. Bradley Knox and Peter Stone. Reinforcement Learning from Simultaneous Human and MDP Reward. In Proceedings of the Eleventh International Conference on Autonomous Agents and Multiagent Systems. June 2012.
pdf
AAMAS 2012

W. Bradley Knox, Cynthia Breazeal, and Peter Stone. Learning from feedback on actions past and intended. In Proceedings of 7th ACM/IEEE International Conference on Human-Robot Interaction, Late-Breaking Reports Session. March 2012.
pdf
HRI 2012

W. Bradley Knox and Peter Stone. Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning. In Proceedings of the Ninth International Conference on Autonomous Agents and Multiagent Systems. May 2010.
Pragnesh Jay Modi Best Student Paper Award
pdf
Poster: [pdf] (2.9 MB)
AAMAS 2010

W. Bradley Knox and Peter Stone. Interactively Shaping Agents via Human Reinforcement: The TAMER Framework. In Proceedings of The Fifth International Conference on Knowledge Capture. September 2009.
pdf
supplemental site
K-CAP09

Gregory Kuhlmann, William B. Knox, and Peter Stone. Know Thine Enemy: A Champion RoboCup Coach Agent. In Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 1463–68, July 2006.
pdf
AAAI 2006

Archival Symposia Papers

Jin Joo Lee, W. Bradley Knox, and Cynthia Breazeal. Modeling the Dynamics of Nonverbal Behavior on Interpersonal Trust for Human-Robot Interactions. In AAAI Spring 2013 Symposium on Trust and Autonomous Systems, March 2013.
pdf

W. Bradley Knox and Peter Stone. Reinforcement Learning from Human Reward: Discounting in Episodic Tasks. In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man). September 2012.
Finalist for CoTeSys Cognitive Robotics Best Paper award
pdf
Ro-Man 2012

W. Bradley Knox, Ian Fasel, and Peter Stone. Design Principles for Creating Human-Shapable Agents. In AAAI Spring 2009 Symposium on Agents that Learn from Human Teachers, March 2009.
pdf
AAAI Spring 2009 Symposium: Agents that Learn from Human Teachers

Workshop Papers 

W. Bradley Knox and James MacGlashan. How to Specify Reinforcement Learning Objectives. RLC 2024 Workshop on Finding the Frame.
pdf

W. Bradley Knox, Samuel Spaulding, and Cynthia Breazeal. Learning Social Interaction from the Wizard: A Proposal. In 3rd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication  (MLIS-2014). July 2014.
pdf
MLIS-2014

W. Bradley Knox, Matthew Taylor, and Peter Stone. Understanding Human Teaching Modalities in Reinforcement Learning Environments: A Preliminary Report. In 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT), July 2011.
pdf
2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT)

W. Bradley Knox and Ole Mengshoel. Diagnosis and Reconfiguration using Bayesian Networks: An Electrical Power System Case Study. In IJCAI 2009 Workshop on Self-* and Autonomous Systems. July 2009.
pdf
IJCAI09-SAS