Stephane Hatgis-Kessell, W. Bradley Knox, Serena Booth, Scott Niekum, Peter Stone. Influencing Humans to Conform to Preference Models for RLHF. Under review.

Calarina Muslimani, Suyog Chandramouli, Serena Booth, W. Bradley Knox, Matthew E. Taylor. Analyzing Reward Functions via Trajectory Alignment. NeurIPS 2024 Workshop on Behavioral ML.
pdf

Juyong Lee, Dongyoon Hahm, June Suk Choi, W. Bradley Knox, Kimin Lee. MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control. arXiv:2410.17520 [cs.LG]. 2024.
arxiv

Michael JQ Zhang, W. Bradley Knox, Eunsol Choi. Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions. arXiv:2410.13788 [cs.CL]. 2024
arxiv

W. Bradley Knox and James MacGlashan. How to Specify Reinforcement Learning Objectives. RLC 2024 Workshop on Finding the Frame.
pdf

Rafael Rafailov, Yaswanth Chittepu, Ryan Park, Harshit Sikchi, Joey Hejna, W. Bradley Knox, Chelsea Finn, Scott Niekum. Scaling laws for reward model overoptimization in direct alignment algorithms. NeurIPS 2024.
pdf
arxiv

Joey Hejna, Rafael Rafailov, Harshit Sikchi, Chelsea Finn, Scott Niekum, W. Bradley Knox, Dorsa Sadigh. Contrastive Preference Learning: Learning from Human Feedback without RL. ICLR 2024. ICLR 2024.
pdf
arxiv

W. Bradley Knox, Stephane Hatgis-Kessell, Sigurdur Orn Adalgeirsson, Serena Booth, Anca Dragan, Peter Stone, and Scott Niekum. Learning Optimal Advantage from Preferences and Mistaking it for Reward. arXiv:2310.02456 [cs.LG]. Oct 2023. AAAI 2024.
pdf
arxiv

W. Bradley Knox*, Stephane Hatgis-Kessell*, Serena Booth, Scott Niekum, Peter Stone, and Alessandro Allievi. Models of human preference for learning reward functions. TMLR 2024.
Spotlight paper at The 5th Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2022)
*Equal contribution
paper page
pdf
arxiv

Serena Booth, W. Bradley Knox, Julie Shah, Scott Niekum, Peter Stone, and Alessandro Allievi. The Perils of Trial-and-Error Reward Design: Misdesign through Overfitting and Invalid Task Specifications. In Proceedings of the Thirty-Seventh National Conference on Artificial Intelligence (AAAI), February, 2023.
Selected for Oral Presentation
pdf

W. Bradley Knox, Alessandro Allievi, Holger Banzhaf, Felix Schmitt, and Peter Stone. Reward (Mis)design for Autonomous Driving. Artificial Intelligence (AIJ), 316, January 2023.
paper page
pdf
AIJ page (open access)
teaser video
Originally published on arxiv in 2021.

Cassidy Curtis, Sigurdur Orn Adalgeirsson, Horia Stefan Ciurdar, Peter McDermott, JD Velásquez, W. Bradley Knox, Alonso Martinez, Dei Gaztelumendi, Norberto Adrian Goussies, Tianyu Liu, and Palash Nandy. Toward Believable Acting for Autonomous Animated Characters. In Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG). November, 2022.
Best Presentation Award (presented by Cassidy Curtis)
pdf
ACM page with video (open access)
MIG 2022
paper website

Yuchen Cui*, Qiping Zhang*, Alessandro Allievi, Peter Stone, Scott Niekum, and W. Bradley Knox. The EMPATHIC Framework for Task Learning from Implicit Human Feedback. In Proceedings of the Conference on Robot Learning (CoRL). November, 2020.
pdf
arXiv
paper website (with talks and links to code)
* Equal contribution

Guangliang Li, Shimon Whiteson, W. Bradley Knox, and Hayley Hung. Social interaction for efficient agent learning from human reward. Autonomous Agents and Multi-Agent Systems (JAAMAS). 32 (1), 1-25. January 2018. (Published online July 2017)
pdf
JAAMAS page (open access)

W. Bradley Knox, Samuel Spaulding, and Cynthia Breazeal. Learning from the Wizard: Programming Social Interaction through Teleoperated Demonstrations. In Proceedings of the Fifteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). May, 2016.
pdf
AAMAS 2016

Emoters, Inc. Application for a 2016 NSF SBIR grant. Design, deployment, and algorithmic optimization of zoomorphic, interactive robot companions. PI: W. Bradley Knox. Awarded December 2016.
pdf

W. Bradley Knox and Peter Stone. Framing Reinforcement Learning from Human Reward: Reward Positivity, Temporal Discounting, Episodicity, and Performance. Artificial Intelligence, 225, 24–50. August 2015.
pdf
Artificial Intelligence Journal page

Guangliang Li, Shimon Whiteson, W. Bradley Knox, and Hayley Hung.  Using informative behavior to increase engagement while learning from human reward. Autonomous Agents and Multi-Agent Systems (JAAMAS). 30 (5), 826–848. September 2016. (Published online August 2015)
pdf
JAAMAS page (open access)

Saleema Amershi*, Maya Cakmak, W. Bradley Knox, and Todd Kulesza. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35 (4): 105-120. December 2014.
pdf
AI Magazine page
*All authors contributed equally

A. Ross Otto, W. Bradley Knox, Art Markman, and  Bradley C. Love. Behavioral and Physiological Signatures of Reflective Exploratory Choice. Cognitive, Affective, & Behavioral Neuroscience. 1-17. March 2014
pdf
CABN Journal page

Guangliang Li, Hayley Hung, Shimon Whiteson, and W. Bradley Knox.  Learning from Human Reward Benefits from Socio-competitive Feedback. In Proceedings of the IEEE International Conference on Development and Learning and on Epigenetic Robots (ICDL-EpiRob). October, 2014.
pdf
ICDL-EpiRob

W. Bradley Knox, Samuel Spaulding, and Cynthia Breazeal. Learning Social Interaction from the Wizard: A Proposal. In 3rd Workshop on Machine Learning for Interactive Systems: Bridging the Gap Between Perception, Action and Communication  (MLIS-2014). July 2014.
pdf
MLIS-2014

W. Bradley Knox, Cynthia Breazeal, and Peter Stone. Training a Robot via Human Feedback: A Case Study. In Proceedings of the International Conference on Social Robotics (ICSR). October, 2013.
Best Paper Award
pdf
talk
ICSR 2013

Guangliang Li, Hayley Hung, Shimon Whiteson, and W. Bradley Knox. Using Informative Behavior to Increase Engagement in the TAMER Framework. In Proceedings of the Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). May, 2013.
pdf
AAMAS 2013

W. Bradley Knox and Peter Stone. Learning Non-Myopically from Human-Generated Reward. In Proceedings of the International Conference on Intelligent User Interfaces (IUI). March 2013.
pdf
IUI 2013

Jin Joo Lee, W. Bradley Knox, Jolie Baumann, Cynthia Breazeal, and David DeSteno. Computationally Modeling Interpersonal Trust. Frontiers in Psychology, 4, 893. 2013.
pdf
Frontiers in Psychology article page

W. Bradley Knox and Peter Stone. Reinforcement Learning from Human Reward: Discounting in Episodic Tasks. In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man). September 2012.
Finalist for CoTeSys Cognitive Robotics Best Paper award
pdf
Ro-Man 2012

W. Bradley Knox, Brian D. Glass, Bradley C. Love, W. Todd Maddox, and Peter Stone. How Humans Teach Agents: A New Experimental Perspective. International Journal of Social Robotics. 4(4) , 409-421. July 2012.
pdf
The final publication is available here at www.springerlink.com.

W. Bradley Knox and Peter Stone. Reinforcement Learning from Simultaneous Human and MDP Reward. In Proceedings of the Eleventh International Conference on Autonomous Agents and Multiagent Systems. June 2012.
pdf
AAMAS 2012

W. Bradley Knox, Cynthia Breazeal, and Peter Stone. Learning from feedback on actions past and intended. In Proceedings of 7th ACM/IEEE International Conference on Human-Robot Interaction, Late-Breaking Reports Session. March 2012.
pdf
HRI 2012

W. Bradley Knox*, A. Ross Otto, Peter Stone, and Bradley C. Love. The Nature of Belief-Directed Exploratory Choice in Human Decision-making. Frontiers in Psychology, 2, 398. 2012.
pdf
Frontiers in Psychology article page
*First two authors contributed equally

W. Bradley Knox, Matthew Taylor, and Peter Stone. Understanding Human Teaching Modalities in Reinforcement Learning Environments: A Preliminary Report. In 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT), July 2011.
pdf
2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers (ALIHT)

W. Bradley Knox and Peter Stone. Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning. In Proceedings of the Ninth International Conference on Autonomous Agents and Multiagent Systems. May 2010.
Pragnesh Jay Modi Best Student Paper Award
pdf
Poster: [pdf] (2.9 MB)
AAMAS 2010

W. Bradley Knox, Ian Fasel, and Peter Stone. Design Principles for Creating Human-Shapable Agents. In AAAI Spring 2009 Symposium on Agents that Learn from Human Teachers, March 2009.
pdf
AAAI Spring 2009 Symposium: Agents that Learn from Human Teachers

W. Bradley Knox and Peter Stone. Interactively Shaping Agents via Human Reinforcement: The TAMER Framework. In Proceedings of The Fifth International Conference on Knowledge Capture. September 2009.
pdf
supplemental site
K-CAP09

Juhyun Lee, W. Bradley Knox, and Peter Stone. Inter-Classifier Feedback for Human-Robot Interaction in a Domestic Setting. Journal of Physical Agents, 2(2):41–50, July 2008. Special Issue on Human Interaction with Domestic Robots
pdf

Gregory Kuhlmann, William B. Knox, and Peter Stone. Know Thine Enemy: A Champion RoboCup Coach Agent. In Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 1463–68, July 2006.
pdf
AAAI 2006

Errata

Honors

• Hasbro Emerging Innovator Award (Finalist; I was lead technologist on the project for Dash Robotics), 2018
• Awarded NSF Small Business Innovation Research (SBIR) grant as PI (application document shared below), 2016
• AI 10 to Watch by IEEE Intelligent Systems, 2013
• Bert Kay Dissertation Award (for best dissertation from UT Austin Computer Science), 2013
• IFAAMAS-12 Victor Lesser Distinguished Dissertation Award (Runner-up), 2013
• ICSR Best Paper Award, 2013
• Ro-Man CoTeSys Cognitive Robotics Best Paper (Finalist), 2012
• AAMAS Pragnesh Jay Modi Best Student Paper Award, 2010
• NSF Graduate Research Fellowship, 2008 – 2011

Academic Organization / Teaching

• 2025 Reinforcement Learning Conference, Scheduling and Keynotes Chair
• 2024 Reinforcement Learning Conference, Program Chair
• 2024 Alignment Workshop in Vienna, Program Chair
• 2022 Workshop on Reinforcement learning with humans in (and around) the loop at RLDM, Co-chair
• 2015 AAAI, Robotics Co-chair
• Interactive Machine Learning course at MIT (2013), Principal lecturer
• 2013 Workshop on Interactive Machine Learning at IUI, Co-chair
• 2012 AAAI Fall Symposium on Robots Learning Interactively from Human Teachers (RLIHT), Co-chair
• 2011 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at IJCAI, Co-chair
• 2010 Workshop on Agents Learning Interactively from Human Teachers (ALIHT) at AAMAS, Co-chair

Research Competitions

Robocup @Home League (home assistant robotics)
2007: 2nd place — Austin Villa @Home Website

Robocup Coach League (opponent modeling)
2006: 2nd place
2005: Champion — Austin Villa 2005 Coach Website