Writing in the Wall Street Journal last week, Robert H. Latiff, a retired Major General in the United States Army now teaching at Notre Dame University, and Patrick J. McCloskey, who teaches at Loyola University in Chicago, take up the troubling question of military drones that, in the near future, will be able to deploy lethal force without direct human control. While acknowledging certain benefits of “emerging robotic armies” (e.g., fewer human casualties for the side deploying the drones), Latiff and McCloskey think that the issues involved are of tremendous moral importance:
The problem is that robotic weapons eventually will make kill decisions with no more than a veneer of human control. Full lethal autonomy is no mere step in military strategy: It will be the crossing of a moral Rubicon. Ceding godlike powers to robots reduces human beings to things with no more intrinsic value than any other object.
When robots rule warfare, utterly without empathy or compassion, humans retain less intrinsic worth than a toaster—which at least can be used for spare parts. . . .
Lethal autonomy also has grave implications for democratic society. The rule of law and human rights depend on an institutional and cultural cherishing of every individual regardless of utilitarian benefit. The 20th century became a graveyard for nihilistic ideologies that treated citizens as human fuel and fodder. . . . Surely death by algorithm is the ultimate indignity.
This makes it sound as if using fully autonomous military drones would be necessarily immoral and, indeed, the moral equivalent of Nazism or Stalinism. While I have great respect for both authors and especially for Gen. Latiff’s service to our country, I think this is confused.
Let’s start at the beginning. A military drone that “make[s] kill decisions” in fact makes no decisions at all, properly speaking. It is a machine, not a moral agent, and no one will hold the drone responsible morally or legally for what it does. On the contrary, if certain human beings build and use military drones that “make their own kill decisions,” these human beings will be morally and legally responsible for the results. This is no surprising conclusion but merely the application of the familiar principle that people are responsible for the reasonably foreseeable consequences of their actions.
Further, it is not true that being killed by a fully autonomous military drone necessarily violates the dignity of the human person. What violates the dignity of the human person is being treated unjustly. Regardless of the physical manner of death, a person who is killed unjustly has his dignity violated; a person who is killed justly does not.
Thus, if a person killed by an autonomous military drone could have been justly killed directly by the hand of a human being, the person killed has not been treated unjustly and his dignity has not been violated merely because the proximate cause of his death was a computer program running on a silicon chip in a drone. In my view, there is no special moral issue involved in the use of fully autonomous military drones.
All that said, I think I see what Latiff and McCloskey are getting at. They are worried that, by setting fully autonomous military drones loose upon the earth, we are valuing too cheaply the human lives that may be taken, primarily those taken by mistake. They worry, for example, that “it is far from clear whether robots can be programmed to distinguish between large children and small adults, and in general between combatants and civilians, especially in urban conflicts.”
This is a serious concern. Although many machines pose dangers to human life (think of automobiles), military drones are designed to kill human beings and so are especially dangerous. Fully autonomous military drones could be even more dangerous because one check on mistaken killing—the human controller—will have been removed. Such machines would thus be extraordinarily dangerous, and Latiff and McCloskey are right to worry about their use.
They are wrong, however, in thinking that there is some foundational moral issue in play. Although the machines involved are extraordinarily dangerous, the moral principle governing their use is perfectly ordinary: It is the familiar one that human beings should engage in an activity that poses dangers to others only if, in the totality of the circumstances, doing so is reasonable—i.e., if the good to be achieved, taking account of the probability of success, is proportionate to the possible ill effects. Whether using fully autonomous military drones is morally permissible will thus turn on the facts of the particular case—e.g., how effective such drones are in killing the enemy, how accurately they distinguish between combatants and non-combatants, and so on.
Consider this parallel. When human beings drive automobiles, they create dangers for innocent third parties, although we think that these dangers are usually justified. But suppose that a company—say Google—develops cars that are driven not by human beings but by computers. (Google is actually working on this, with significant success.) Suppose further that it turns out that computers can drive cars more safely than human beings can (which is also likely to be the case someday). So if we all switch to computer-driven cars, the total accident rate will go down, and fewer people will be killed in car crashes.
Of course, some accidents will still happen, and some people will still be killed. Would anyone really say that those killed by computer-driven cars have been harmed more than those killed by the mistakes of human drivers? I don’t see any plausible basis for such a position.
And something analogous may turn out to be true about fully autonomous military drones. As is well known, human soldiers make mistakes distinguishing between combatants and non-combatants and thus sometimes kill the innocent. Fully autonomous military drones will make such mistakes too. The question is which set of mistakes is worse—that is, who will have the higher error rate, the human beings or the drones.
The answer to that could well vary with the context. In any event, it is an empirical question, an empirical question the resolution of which will figure in the moral determination of whether our using such extraordinarily dangerous machines is morally justified in particular circumstances. That will be a very difficult question, but it will be difficult because it will turn on facts and circumstances that we can know only imperfectly, not because it involves foundational issues in moral philosophy.
Robert T. Miller is professor of law and F. Arnold Daum Fellow in corporate law at the University of Iowa.
Become a fan of First Things on Facebook, subscribe to First Things via RSS, and follow First Things on Twitter.
Comments:
We might as well be up front about it: this is sheer consequentialism. And we should not be surprised that consequentialists overlook entirely the "foundational issues in moral philosophy."
The ever-increasing reliance on technology makes us more vulnerable to
asymmetrical warfare strategies, for which we are ill-equipped. Additionally, drones cannot conquer and hold territory, so boots on the ground will always be required for victory.
The danger is that an over-reliance on drones makes killing easier, but with less strategic purpose and importance. It will become a trivial matter to send drones off to kill, yet gaining the political ends desired will be more difficult to accomplish. As Robert E. Lee said at Fredricksburg, "It is well that war is so terrible - otherwise we would grow too fond of it." Making war less terrible through robot drones may lead decision-makers to use (and abuse) that warfighting option. Still, as William T. Sherman wrote to those in Atlanta, "War is cruelty, and you cannot refine it" and this is true whether in hand-to-hand combat, or through the use of robot drones.
The problem with deciding criminal cases with algorithms is that many cases involve judgments of human motivation and behavior that are far too subtle for any algorithm to handle --- certainly any algorithm devised by man. On the other hand, judging whether someone is firing rockets at you may not be beyond the capacities of an algorithm.
It also should be noted that algorithms are used at many levels in determining guilt or innocence. Applying the criterion "means, motive, and opportunity" is an algorithm. When a jury member makes a judgment that someone is lying on the witness stand based on body language, facial expressions, tone of voice, and so on, he is unconsciously using very sophisticated algorithms for interpreting such things that nature has provided him with. There are algorithms that determine whether fingerprints match. Examples could be multiplied. For human beings to entrust certain tasks to algorithms, when those tasks are much more accurately done by algorithms is not intrinsically immoral. And therefore consequentialism has nothing to do with this.
The best science fiction has always addressed the moral difficulties of disruptive technology, and has proven prescient. It didn't take the makers of the popular game "Call of Duty: Black Ops II" much to imagine a future wherein the robotic legions are hacked by a nefarious mastermind and made to prey on the innocent.
While human beings can be subjected to torture and coercion to try and compel them to immoral actions, such efforts are largely time-consuming, expensive, and as testified to in the lives of thousands of Christian martyrs, often ultimately fail.
The virtues of the builders/programmers of a drone army are, in the final analysis, IRRELEVANT, and not just because of the Law of Unintended Consequences (delved into briefly by the authors). The true danger of roboticizing the military is the ease of the its co-option. One sufficiently brilliant person of evil intent can render--at mere keystrokes--the good efforts and intentions of many. The weekly viral e-mail should be sufficient enough a reminder of the dangers, and shame on those who would choose expediency at the cost of human life...
Here are two (2), very well-produced, videos that anchor my points:
Emmy Award-winning newscaster Shad Olson’s ‘The Great Drone Debate’, featuring US Senator John Thune (7:41):
http://www.youtube.com/watch?v=ssoOASanKao
Here’s a mind-blowing, well-done animated short that really captures our collective angst that if the road to perdition is paved with good intentions, then domestic drones are a superhighway to an Orwellian panoptic gulag (3:22):
http://vimeo.com/59689349
For national security purposes, Americans are already subject to warrantless wiretaps of calls and emails, the warrantless GPS “tagging” of their vehicles, the domestic use of Predators or other spy-in-the-sky drones, and the Department of Homeland Security’s monitoring of all our behavior through “data fusion centers.”
http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/
America’s promise has always been the power of the many to rule, instead of the one. Ungoverned drone usage, particularly domestically, gives power to the one.
No one denies that, in general, innocent people should not be judged guilty. The question is whether there is a qualitative difference in moral agency when one man condemns another to death and when a computer recommends the same—whether the replacement of man by machine touches on fundamental moral issues. And of course it does: with a human jury, representatives from one's community publicly states that the defendant deserves death, while the latter merely produces a calculation. The relevant moral agents are in the courtroom, face the accused, and take a very direct responsibility for his fate. A computer programmer may have some responsibility for his algorithm’s computation, but of an obviously different sort.
Which brings us to the second point. The human mind, as Hubert Dreyfus famously pointed out in Why Computers Can’t Think, does not operate according to formulae, but is embedded in practical contexts in all their indeterminacy and ambiguity. The claim that jurors “unconsciously us[e] very sophisticated algorithms” to evaluate juror testimony invokes a theory of mind and perception that has been deservedly discreditable for decades. This point is even more salient in the context of moral judgment than artificial intelligence. As the work of thinkers as diverse as Merleu-Ponty, Searle, and MacIntyre has shown, moral agency, thought, and biology are intimately bound together.
So claim that “consequentialism has nothing to do with this” is rather obviously incorrect. Only if one assumes that the consequences of an action are of primary moral relevance can one assert that the manner in which the action is carried out, the actual relation of agent to patient, etc., do not raise fundamental moral issues, and to further overlook the importance of practical coping, social context, and biology for human reasoning makes things worse.
While I certainly agree that the human mind does things that cannot be explained algorithmically --- as I argue at great length in my own book (Modern Physics and Ancient Faith), there are some things that it does do mechanically. That has not been discredited. E.g. recognizing faces, or riding a bicycle while one's attention is on something else, or even walking while one is engaged in conversation. (I do not distinguish here between programs and neural networks.)
When we recognize faces, or read someone's emotions from body language, we are (for the most part) doing the kinds of things that even many animals do, and in some cases responding to the same kinds of cues.
There are many cases where human beings put in place essentially mechanical procedures that cause actions to be taken that have consequences for people's lives. For example, there are a variety of mechanisms that react to sudden emergencies where there is no time for human deliberation. Whether it is reasonable and justified to put such mechanisms in place depends on an estimation of relative risks (among other things). The moral responsibility lies with the people who put those systems in place.
Finally, to say that the morality of *some* kinds of decisions depends on the consequences is not consequentialism.
In fact, my great fear as a professional teacher is that it is just a matter of time before we see drones used by private citizens to engage in school shootings. About $700 and some technical knowledge already makes it possible to mount a firearm on a commercially available quadrotor ($300) robot that can navigate inside buildings...
Development does not, however, require that we at any point cede kill decisions to said drones. Isn't one of the primary points of developing them that they are expendable? Why write an algorithm that would allow a drone to initiate lethal force against a human seeking to destroy it? The drone feels no fear of death--one advantage over a human soldier that could limit bloodshed. Let it die. Endow it with protective responses. But it shouldn't ever have authorization to autonomously kill.
And killing in a phalanx was no more "hard" then killing on a screen. It was more difficult physically. But any given warrior saw his enemies, if at all, as faceless foes dressed like Imperial Stormtroopers. He was propelled forward by a herd of men and had less individual decision about killing then a drone pilot. The assumption that past wars were somehow more "personal" is flawed. Past wars were about formations fighting other formations both of whom happened to contain human beings. A formation is just as dehumanized as a dot on a screen.
Morality about war is not a function of technology.
I agree that morality about war is not a function of technology. My point was that technology has tended to gloss over (or obscure) the moral decisions to be made as the "pointy end of the spear" was moved farther away from the decision-makers. I know from personal experience that there is a different mind set when generating lists of targets over the horizon rather than those directly in front of friendly forces. Even in the few decades since I first joined the military the techological changes have greatly influenced the decision-making relative to combat.
Your description of hoplite warriors (Imperial Stormtroopers?) short-changed the courage, skill and fortitude necessary to perform successfully in that arena. There were decisions to be made (the timing of the thrust and its location, etc.) and once initial contact was made, the follow-up combat usually involved finding individual soldiers to attack or defend oneself from.
Given the number of general and regimental officers killed with their troops during the Civil War, formations were still pretty human.
The question was never someone bears moral responsibility for drone strikes, but rather whether automating the decision to kill persons raises “foundational moral questions.” I have not claimed that automated drones are immoral, only that the manner in which one goes about a course of activity—in this case changing the nature of battle so that certain combatants do not have to actually see those they kill, are not in danger themselves, and do not even have to make a particular decision that results in the death of another but can automate that process, etc.—raises fundamental moral issues.
Unless, that is, one thinks that the manner in which an action is carried out does not raise basic moral questions, so long as the result (the death of terrorists) is justified. Indeed, Mr. Miller explicitly uses consequentialist reasoning in the article; his test is whether “the good to be achieved … is proportionate to the possible ill effects.” It boils down to—his words—which has the “higher error rate.” That’s not an arateic or a deontological approach (indeed, these are implicitly excluded); it explicitly turns on consequences, rather than the way in which the consequences are effected.
As this argument, in principle, applies to other decisions to take a human life, I thought the absurdity would be obvious with the computerized jury example. But alas, what does one do when one’s reductio is accepted with such alacrity? If one cannot see foundational moral issues arising from the replacement of a human decision to take a particular human life to that of a computer algorithm—whatever its ultimate justifiability—what example would be shock that person into moral perceptivity? If automating the decision to kill does not raise foundational moral questions, what does?
For example, a pilot in pursuit of an enemy plane locks onto the aircraft with his fire control radar. He can't see the plane with his eyes, but can be reasonably certain that it is a legitimate target. He launches a radar-controlled missile, and the computers take over, guiding the missile in for the kill. Did the computers kill the enemy pilot? Well, in a sense they did, but the pilot was the actual decision-making agent, and bears ultimate responsibility. As the authors above pointed out, the ultimate moral responsibility will likewise lie with the programmers and unit commanders behind the "autonomous" drone. The scary aspect of this lies in the idea that drones may become amoral agents with a will of their own, and that is not something I believe machines can ever become.
When I was a B-52 crew member, long ago, I worked on a weapons system that was still under complete human control. The human agents, though, were prepared to drop bombs on cities that, in the case of the larger-yield weapons, had a 100% kill radius of about ten miles. That is, most of the citizens of a large metropolitan region would eventually die. There was no pretense of distinguishing between combatants and non-combatants. Compared to a moral nightmare like that, I'd have to say that the drones seem fairly innocuous.