The First Law of Robotics (a call to arms)

摘要

Even before the advent of Artificial Intelligence, sci-ence fiction writer Isaac Asimov recognized that an agent must place the protection of humans from harm at a higher priority than obeying human orders. In-spired by Asimov, we pose the following fundamental questions: (1) How should one formalize the rich, but informal, notion of “harm”? (2) How can an agent avoid performing harmful actions, and do so in a com-putationally tractable manner? (3) How should an agent resolve conflict between its goals and the need to avoid harm? (4) When should an agent prevent a human from harming herself? While we address some of these questions in technical detail, the primary goal of this paper is to focus attention on Asimov’s concern: society will reject autonomous agents unless we have some credible means of making them safe! The Three Laws of Robotics: A robot may not injure a human being, or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the