|
LIFEBOAT FOUNDATION SPECIAL REPORT
LIFEBOAT FOUNDATION SPECIAL REPORT
THE AGE OF VIRTUOUS MACHINES
By Lifeboat Foundation Scientific Advisory Board member J. Storrs "Josh"
Hall.
Print report!
In the "hard takeoff" scenario, a psychopathic AI suddenly emerges at a
superhuman level, achieving universal dominance. Hall suggests an
alternative: we've gotten better because we've become smarter, so AIs
will evolve "unselfish genes" and hyperhuman morality. More honest,
capable of deeper understanding, and free of our animal heritage and
blindnesses, the children of our minds will grow better and wiser than
us, and we will have a new friend and guide — if we work hard to
earn the
privilege of associating with them.
This is chapter 20 of
Beyond AI: Creating the Conscience of the Machine.
"To you, a robot is a robot. Gears and metal. Electricity and
positrons. Mind and iron! Human-made! If necessary, human-destroyed.
But you haven't worked with them, so you don't know them. They're a
cleaner, better breed than we are."
Ethical
AIs
Over the past decade, the concept of a technological singularity has
become better understood. The basic idea is that the process of creating
AI and other technological change will be accelerated by AI itself, so
that sometime in the coming century the pace of change will become so
rapid that we mere mortals won't be able to keep up, much less control
it.
British statistician and colleague of Turing I. J.
Good wrote in
1965, "Let an ultraintelligent machine be defined as a machine that can
far surpass all the intellectual activities of any man however clever.
Since the design of machines is one of these intellectual activities, an
ultraintelligent machine could design even better machines; there would
then unquestionably be an "intelligence explosion", and the intelligence
of man would be left far behind." The disparate intellectual threads,
including the word "singularity" from which the modern concept is woven,
were pulled together by Vernor Vinge in 1993. More recently it was the
subject of a best-selling book by Ray Kurzweil. There is even a
reasonably well-funded think tank, the Singularity Institute for
Artificial Intelligence (SIAI), whose sole concern is singularity
issues.
It is common (although not universal) in Singularity studies to worry
about autogenous AIs. The SIAI, for example, makes it a top concern,
whereas Kurzweil is more sanguine that AIs will arise by progress along
a path enabled by neuroscience and thus be essentially human in
character. The concern, among those who share it, is that epihuman AIs
in the process of improving themselves might remove any conscience or
other constraint we program into them, or they might simply program
their successors without them.
But it is in fact we, the authors of the first AIs, who stand at the
watershed. We cannot modify our brains (yet) to alter our own
consciences, but we are faced with the choice of building our creatures
with or without them. An AI without a conscience, by which I mean both
the innate moral paraphernalia in the mental architecture and a
culturally inherited ethic, would be a superhuman
psychopath.
Prudence, indeed, will dictate that superhuman psychopaths should not be
built; however, it seems almost certain someone will do it anyway,
probably within the next two decades. Most existing AI research is
completely pragmatic, without any reference to moral structures in
cognitive architectures. That is to be expected: just getting the darn
thing to be intelligent is as hard a problem as we can handle now, and
there is time enough to worry about the brakes after the engine is
working.
As I noted before, much of the most advanced
research is
sponsored by the military or corporations. In the military, the notion
of an autonomous machine being able to question its orders on moral
grounds is anathema. In corporate industry, the top goal seems likely to
be the financial benefit of the company. Thus, the current probable
sources of AI will not adhere to a universally adopted philanthropic
formulation, such as Asimov's Three Laws. The reasonable assumption then
is that a wide variety of AIs with differing goal structures will appear
in the coming decades.
Hard
Takeoff
Within thirty years, we will have the technological means to create
superhuman intelligence. Shortly after, the human era will be
ended.
Vernor Vinge, 1993
A subtext of the singularitarian concern is there may be the possibility
of a sudden emergence of (a psychopathic) AI at a superhuman level, due
to a positive feedback in its autogenous capabilities. This scenario is
sometimes referred to as a "hard takeoff". In its more extreme
versions,
the concept is that a hyperhuman AI could appear virtually overnight and
be so powerful as to achieve universal dominance. Although the scenario
usually involves an AI rapidly improving itself, it might also happen by
virtue of a longer process kept secret until sprung on the world, as in
the movie
Colossus: The Forbin Project.
The first thing that either version of the scenario requires is the
existence of computer hardware capable of running the hyperhuman AI. By
my best estimate, hardware for running a
diahuman AI
(somewhere across the broad range of human capability)
currently exists,
but is represented by the top ten or so supercomputers in the world.
These are multimillion-dollar installations, and the dollars were not
spent to do AI experiments. And even if someone were to pay to dedicate,
say, an IBM blue gene or Google's fabled grid of stock PCs to running an
AI full-time, they would only approximate a normal human intelligence.
There would have to be a major project to build the hardware of a
seriously epihuman, much less hyperhuman, AI with current computing
technology.
Second, even if the hardware were available, the software is not. The
fears of a hard takeoff are based on the notion that an early
superintelligence would be able to write smarter software faster for the
next AI, and so on. It does seem likely that a properly structured AI
could be a better programmer than a human of otherwise comparable
cognitive abilities, but remember that as of today, automatic
programming remains one of the most poorly developed of the AI
subfields. Any reasonable extrapolation of current practice predicts
that early human-level AIs will be secretaries and truck drivers, not
computer science researchers or even programmers.
Even
when a diahuman
AI computer scientist is achieved, it will simply add one more scientist
to the existing field, which is already bending its efforts toward
improving AI. That won't speed things up much. Only when the total AI
devoting its efforts to the project begins to rival the intellectual
resources of the existing human AI community — in other words,
being
already epihuman — will there be a really perceptible acceleration.
We are
more likely to see an acceleration from a more prosaic source first:
once AI is widely perceived as having had a breakthrough, it will
attract more funding and human talent.
Third, intelligence does not spring fully formed like Athena from the
forehead of Zeus. Even we humans, with the built-in processing power of
a supercomputer at our disposal, take years to mature. Again, once
mature, a human requires about a decade to become really expert in any
given field, including AI programming. More to the point, it takes the
scientific community some extended period to develop a theory, then the
engineering community some more time to put it into
practice.
Even if we
had a complete and valid theory of mind, which we do not, putting it
into software would take years; and the early versions would be
incomplete and full of bugs. Human developers will need years of
experience with early AIs before they get it right. Even then they will
have systems that are the equivalent of slow, inexperienced humans.
Advances in software, similar to Moore's law for hardware, are less
celebrated and less precisely measurable, but nevertheless real.
Advances in algorithmics have tended to produce software speedups
roughly similar to hardware ones. Running this backward, we can say that
the early software in any given field is much less efficient than later
versions. The completely understood, tightly coded, highly optimized
software of mature AI may run a human equivalent in real time on a 10
teraops machine. Early versions will not.
There are two wild-card possibilities to consider. First, rogue AIs
could be developed using botnets, groups of hijacked PCs communicating
via the Internet. These are available today from unscrupulous hackers
and are widely used for sending spam and conducting Ddos attacks on Web
sites. A best estimate of the total processing power on the Internet
runs to 10,000 Moravec HEPP or 10 Kurzweil HEPP, although it is unlikely
that any single coordinated botnet could collect even a fraction of 1
percent of that at any given time.
Moreover, the extreme
forms of
parallelism needed to use this form of computing, along with the
communication latency involved, will tend to push the reasonable
estimates toward the Kurzweil level (which is based on the human brain
with its high-parallelism, slow-cycle time architecture). That, together
with the progress of the increasingly sophisticated Internet security
community, will make the development of AI software much harder in this
mode than in a standard research setting. The "researchers" would have
to worry about fighting for their computing resources as well as
figuring out how to make the AI work — and the AI, to be able to
extend
their work, would have to do the same. Thus, while we can expect botnet
AIs in the long run, they are unlikely to be first.
The second wild-card possibility is that Marvin Minsky is right. Almost
every business and academic computing facility offers at least a Minsky
HEPP. If an AI researcher found a simple, universal learning algorithm
that allowed strong positive feedback into such a highly optimized form,
it would find ample processing power available. And this could be
completely aboveboard? Minsky HEPP costs much less than a person is
worth, economically.
Let me, somewhat presumptuously, attempt to explain Minsky's intuition
by an analogy: a bird is our natural example of the possibility of
heavier-than-air flight. Birds are immensely complex: muscles, bones,
feathers, nervous systems. But we can build working airplanes with
tremendously fewer moving parts. Similarly, the brain can be greatly
simplified, still leaving an engine capable of general conscious
thought.
My own intuition is that Minsky is closer to
being right than
is generally recognized in the AI community, but computationally
expensive heuristic search will turn out to be an unavoidable element of
adaptability and autogeny. This problem will extend to any AI capable of
the runaway feedback loop that singularitarians fear.
Moral
Mechanisms
It is therefore most likely that a full decade will elapse between the
appearance of the first genuinely general, autogenous AIs and the time
they become significantly more capable than humans. This will indeed be
a crucial period in history, but no one person, group, or even school of
thought will control it.
The question instead is, what can
be done to
influence the process to put the AIs on the road to being a stable
community of moral agents? A possible path is shown in Robert Axelrod's
experiments and in the original biological evolution of our own
morality. In a world of autonomous agents who can recognize each other,
cooperators can prosper and ultimately form an evolutionarily stable
strategy.
Superintelligent AIs should be just as capable of understanding this as
humans are. If their environment were the same as ours, they would
ultimately evolve a similar morality; if we imbued them with it in the
first place, it should be stable. Unfortunately, the environment they
will inhabit will have some significant differences from
ours.
The Bad
News
Inhomogeneity
The disparities among the abilities of AIs could be significantly
greater than those among humans and more correlated with an early "edge"
in the race to acquire resources. This could negate the evolutionary
pressure to reciprocal altruism.
Self-Interest
Corporate AIs will almost certainly start out self-interested, and
evolution favors effective self-interest. It has been suggested by
commentators such as Steven Pinker,
Eliezer Yudkowsky, and Jeff Hawkins,
that AIs would not have the "baser" human instincts built in and thus
would not need moral restraints. But it should be clear they could be
programmed with baser instincts, and it seems likely that corporate ones
will be aggressive, opportunistic, and selfish, and that military ones
will be programmed with different but equally disturbing motivations.
Furthermore, it should be noted that any goal structure implies
self-interest. Consider two agents, both with the ability to use some
given resource. Unless the agents' goals are identical, each will
further its own goal more by using the resource for its own purposes and
consider it at best suboptimal and possibly counterproductive for the
resource to be controlled and used by the other agent toward some other
goal. It should go without saying that specific goals can vary wildly
even if both agents are programmed to seek, for example, the good of
humanity.
The Good
News
Intelligence Is Good
"There is but one good, namely, knowledge; and but one evil, namely
ignorance."
As a matter of practical fact, criminality is strongly and negatively
correlated with IQ in humans. The popular image of the tuxedo-wearing,
suave jet-setter jewel thief to the contrary notwithstanding, almost all
career criminals are of poor means as well as of lesser intelligence.
Nations where the rule of law has broken down are poor compared to more
stable societies. A remarkable document published by the World Bank in
2006 surveys the proportions of natural resources, produced capital
(such as factories and roads), and intangible capital (education of the
people, value of institutions, rule of law, likelihood of saving without
theft or confiscation). Here is a summary. Note that the wealth column
is total value, not income.
| Income Group | Wealth per Capita |
Natural Resources | Produced Capital |
Intangible Capital |
| Low
Income | $7,532 | $1,925 | $1,174 | $4,434 |
| Medium
Income | $27,616 | $3,496 | $5,347 | $18,773 |
| High Income | $439,063 | $9,531 | $76,193 |
$353,339 |
In a wealthy country, natural resources such as farmland are worth more
but only by a small amount, mostly because they can be more efficiently
used. The fraction of total wealth contributed by natural resources in a
wealthy country is only 2 percent, as compared to 26 percent in a poor
one. The vast majority of the wealth in high-income countries is
intangible: it is further broken down by the report to show that roughly
half of it represents people's education and skills, and the other half
the value of the institutions?n other words, the opportunities the
society gives its citizens to turn efforts into value.
Lying, cheating, and stealing are profitable only in the very short
term. In the long run, honesty is the best policy; leaving cheaters
behind and consorting with other honest creatures is the best plan. The
smarter you are, the more likely you are to understand this and to
conduct your affairs accordingly.
Original Sin
"We have met the enemy, and he is us!"
Porkypine in Walt Kelly's
Pogo
Developmental psychologists have sobering news for humankind, which
echoes and explains the old phrase, "Someone only a mother could love."
Simply put, human babies are born to lie, cheat, and steal. As Matt
Ridley put it in another connection, "Vervet monkeys, like
two-year-olds, completely lack the capacity for empathy." Law and custom
recognize this as well: children are not held responsible for their
actions until they are considerably older than two.
In fact, recent neuroscience research using brain scans indicates that
consideration for other people's feelings is still being added to the
mental planning process up through the age of twenty.
Children are socialized out of the condition we smile at and call
"childishness" (but think how differently we'd refer to an adult who
acted, morally, like a two-year-old). Evolution and our genes cannot
predict what social environment children will have to cope with, so they
make children ready for the rawest and nastiest; they can grow out of it
if they find themselves in civilization, but growing up mean is your
best chance of survival in many places.
With AIs, we can simply reverse the default orientation: AIs can start
out nice, then learn the arts of selfishness and revenge only if the
situation demands it.
Unselfish Genes
Reproduction of AIs is likely to be completely different from that of
humans. It will be much simpler just to copy the program. It seems quite
likely that ways will be found to encode and transmit concepts learned
from experience more efficiently than we do with language. In other
words, AIs will probably be able to inherit acquired characteristics and
to acquire substantial portions of their mentality from others in a way
reminiscent of bacteria exchanging plasmids.
For these reasons, individual AIs are likely to be able to have the
equivalent of both memories and personal experience stretching back in
time before they were "born", as experienced by many other AIs. To the
extent that morality is indeed a summary encoding of lessons learned the
hard way by our forebears, AIs could have a more direct line to it. The
superego mechanisms by which personal morality trumps common sense
should be less necessary, because the horizon effect for which it's a
heuristic will recede with wider experience and deeper understanding.
At the same time, AIs will lack some of the specific pressures, such as
sexual jealousy, that we suffer from because of the sexual germ-line
nature of animal genes. This may make some of the nastier features of
human psychology unnecessary.
Cooperative Competition
For example, AIs could well be designed without the mechanism we seem to
have whereby authority can short-circuit morality, as in the
Milgram
experiments1. This is the equipment that
implements the
distributed
function of the pecking order. The pecking order had a clear, valuable
function in a natural environment where the Malthusian dynamic held
sway: in hard times, instead of all dying because evenly divided
resources were insufficient, the haves survived and the have-nots were
sacrificed. In order to implement such a stringent function without
physical conflict that would defeat its purpose, some very strong
internal motivations are tied to perceptions of status, prestige, and
personal dominance.
1. In 1963 psychologist Stanley Milgram did some famous
experiments to
test the limits of people's consciences when under the influence of an
authority figure. The shocking results were that ordinary people will
inflict torture on others simply because they were told to do so by a
scientist in a lab coat.
On the one hand, the pecking order is probably responsible for saving
humanity from extinction numerous times. It forms a large part of our
collective character, will we or nill we. AIs without pecking-order
feelings would see humans as weirdly alien (and we them).
On the other, the pecking order short-circuits our moral sense. It
allows political and religious authority figures to tell us to do
hideously immoral things that we would be horrified to do in other
circumstances. It makes human slavery possible as a stable form of
social organization, as is evident throughout many centuries of history.
And what's more, it's not necessary. Market economics is much better at
resource allocation than the pecking order. The productivity of
technology is such that the pecking order's evolutionary premise no
longer holds. Distributed learning algorithms, such as the scientific
method or idea futures markets, do a better job than the judgment of a
tribal chieftain.
Comparative Advantage
The economic law of comparative advantage states that cooperation
between individuals of differing capabilities remains mutually
beneficial. Suppose you are highly skilled and can make eight widgets
per hour or four of the more complicated doohickeys. Your neighbor Joe
does everything the hard way and can make one widget or one doohickey in
an hour. You work an eight-hour day and produce sixty-four widgets, and
Joe makes eight doohickeys. Then you trade him twelve widgets for the
eight doohickeys. You get fifty-two widgets and eight doohickeys total,
which would have taken you an extra half an hour in total to make
yourself; and he gets twelve widgets, which he would have taken four
extra hours to make!
In other words, even if AIs become much more productive than we are, it
will remain to their advantage to trade with us and to ours to trade
with them.
Unlimited Lifetime
"And behold joy and gladness, ... eating flesh, and drinking wine:
let us
eat and drink; for to morrow we shall die."
People have short-term planning horizons in many cases. Human mortality
not only puts a limit on what we can reasonably plan for the future, but
our even shorter-lived ancestors passed on genes that shortchange even
that in terms of the instinctive (lack of) value we put on the future.
The individual lifetime of an AI is not arbitrarily limited. It has the
prospect of living into the far future, in a world whose character its
actions help create. People begin to think in longer range terms when
they have children and face the question of what the world will be like
for them. An AI can instead start out thinking about what the world will
be like for itself and for any copies of itself it cares to make.
Besides the unlimited upside to gain, AIs will have an unlimited
downside to avoid: forever is a long time to try to hide an illicit deed
when dying isn't the way you expect to escape retribution.
Broad-based Understanding
Epihuman, much less hyperhuman AIs will be able to read and absorb the
full corpus of writings in moral philosophy, especially the substantial
recent work in evolutionary ethics, and understand it better than we do.
They could study game theory — consider how much we have learned in
just
fifty years! They could study history and economics.
E. O. Wilson has a compelling vision of consilience, the unity of
knowledge. As we fill in the gaps between our fractured fields of
understanding, they will tend to regularize and correct one another. I
have tried to show in a small way how science can come to inform our
understanding of ethics. This is but the tiniest first step. But that
step shows how much ethics is a key to anything else we might want to do
and is thus as worthy of study as anything else.
Keep a Cool Head
"Violence is the last refuge of the incompetent."
Remember Newcomb's Problem, the game with the omniscient being (or team
of psychologists) and the million- and thousand-dollar boxes. It's the
one that, in order to win it, you have to be able to "chain yourself
down" in some way so you can't renege at the point of choice. For this
purpose, evolution has given humans the strong emotions.
Thirst for revenge, for example, is a way of guaranteeing any potential
wrongdoers that you will make any sacrifice to get them back, even
though it may cost you much and gain you nothing to do so. Here, the
point of choice is after the wrong has been done — you are faced
with an
arduous, expensive, and quite likely dangerous pursuit and attack on the
offender; rationally, you are better off forgetting it in many cases. In
the lawless environment of evolution, however, a marauder who knew his
potential victims were implacable revenge seekers would be deterred. But
if there is a police force this is not as necessary, and the emotion to
get revenge at any cost can be counterproductive.
So collective arrangements like police forces are a significantly better
solution. There are many such cases where strong emotions are
evolution's solution to a problem, but we have found better ones. AIs
could do better yet in some cases: the solutions to Newcomb's Problem
involving Open Source-like guarantees of behavior are a case in point.
In addition, the lack of the strong emotions can be beneficial in many
cases. Anger, for example, is more often a handicap than a help in a
world where complex interactions are more common than physical
altercations. A classic example is poker, where the phrase "on tilt" is
applied to a player who become frustrated and loses his cool analytical
approach. A player "on tilt" makes aggressive plays instead of optimal
ones and loses money.
With other solutions to Newcomb's Problem available, AIs could avoid
having strong emotions, such as anger, with their concomitant
infelicities.
Mutual Admiration Societies
Moral AIs would be able to track other AIs in much greater detail than
humans do one another and for vastly more individuals. This allows a
more precise formation and variation of cooperating
groups.
Self-selecting communities of cahooting AIs would be able to do the same
thing that tit-for-tat did in Axelrod's tournaments: prosper by virtue
of cooperating with other "nice" individuals. Humans, of course, do the
same, but AIs would be able to do it more reliably and on a larger
scale.
A Cleaner, Better Breed
"Reflecting on these questions, I have come to a conclusion which,
however implausible it may seem on first encounter, I hope to leave the
reader convinced: not only could an android be responsible and culpable,
but only an android could be."
Joseph Emile Nadeau
AIs will (or at least could) have considerably better insight into their
own natures and motives than humans do. Any student of human nature is
well aware how often we rationalize our desires and actions. What's
worse, it turns out that we are masters of self-deceit: given our
affective display subsystems, the easiest way to lie undetectably is to
believe the lie you're telling! We are, regrettably, very good at doing
exactly that.
One of the defining characteristics of the human mind has been the
evolutionary arms race between the ability to deceive and the ability to
penetrate subterfuge. It is all too easy to imagine this happening with
AIs (as it has with governments — think of the elaborate spying and
counterspying during the cold war). On the other hand, many of the other
moral advantages listed above, including Open-Source honesty and longer
and deeper memories could well mean that mutual honesty societies might
be a substantially winning strategy.
Thus, an AI may have the ability to be more honest than humans, who
believe our own confabulations.
Invariants
A Class of Photometric Invariants:
Separating Material from Shape and Illumination
How can we know that our AIs will retain the good qualities we give them
once they have improved themselves beyond recognition in the far future?
Our best bet is a concept from math called an invariant —
property of
something that remains the same even when the thing itself changes. We
need to understand what desirable traits are likely to be invariant
across the process of radical self-improvement, and start with those.
Knowledge of economics and game theory are likely candidates, as is
intelligence itself. An AI that understands these things and their
implications is unlikely to consider forgetting them an improvement. The
ability to be guaranteeably trustworthy is likewise valuable and
wouldn't be thrown away. Strong berserker emotions are clearly not a
smart thing to add if you don't have them (and wouldn't form the
behavior guarantees that they do in humans anyway, since the
self-improving AI could always edit them out!), so lacking them is an
invariant where usable alternatives exist.
Self interest is another property that is typically invariant with, or
indeed reinforced by, the evolutionary process. Surprisingly, however,
even though I listed it with the bad news above, it can form a
stabilizing factor in the right environment. A non-self-interested
creature is hard to punish; its actions may be random or purely
destructive. With self-interest, the community has both a carrot and a
stick. Enlightened self-interest is a property that can be a
beneficial
invariant.
If we build our AIs with these traits and look for others like them, we
will have taken a strong first step in the direction of a lasting
morality for our machines.
Artificial Moral
Agency
A lamentable phenomenon in AI over the years has been the tendency for
researchers to take almost laughably simplistic formal systems and claim
they implemented various human qualities or capabilities. In many cases
the ELIZA Effect aligns with the hopes and the ambitions of the
researcher, clouding his judgment. It is necessary to reject this
exaggeration firmly when considering consciousness and free will. The
mere capability for self-inspection is not consciousness; mere
decision-making ability is not free will.
We humans have the strong intuition that mentalistic properties we
impute to one another, such as the two above, are essential ingredients
in whatever it is that makes us moral agents — beings who have real
obligations and rights, who can be held responsible for their actions.
The ELIZA Effect means that when we have AIs and robots acting like they
have consciousness and free will, most people will assume that they do
indeed have those qualities, whatever they are. The problem, to the
extent that there is one, is not that people don't allow the moral
agency of machines where they should but that they anthropomorphize
machines when they shouldn't.
I've argued at some length that there will be a form of machine,
probably in the not-too-distant future, for which an ascription of moral
agency will be appropriate. A machine that is conscious to the extent
that it summarizes its actions in a unitary narrative and that has free
will to the extent that it weighs its future acts using a model informed
by the narrative will act like a moral agent in many ways; in
particular, its behavior will be influenced by reward and punishment.
There is much that could be added to this basic architecture, such as
mechanisms to produce and read affective display, and things that could
make the AI a member of a memetic community: the love of trading
information, of watching and being watched, of telling and reading
stories. These extend the control/feedback loops of the mind out into
the community, making the community a mind writ large. I have talked
about the strong emotions and how in many cases their function could be
achieved by better means.
Moral agency breaks down into two parts — rights and
responsibility — but they
are not coextensive. Consider babies: we accord them rights but not
responsibilities. Robots are likely to start on the other side of that
inequality, having responsibilities but not rights, but, like babies, as
they grow toward (and beyond) full human capacity, they will aspire to
both.
Suppose we consider a contract with a potential AI: "If you'll work for
me as a slave, I'll build you." In terms of the outcome, there are three
possibilities: it doesn't exist, it's a slave, or it's a free creature.
By offering it the contract, we give it the choice of the first two.
There are the same three possibilities with respect to a human slave: I
kill you, I enslave you, or I leave you free. In human terms, only the
last is considered moral.
In fact, many (preexisting) people have chosen slavery instead of
nonexistence. We could build the AI in such a way to be sure that it
would agree, given the choice. In the short run, we may justify our
ownership of AIs on this ground. Corporations are owned, and no one
thinks of a corporation as resenting that fact.
In the long run, especially once the possibility of responsible free AIs
is well understood, there will inevitably be analogies made to the human
case, where the first two possibilities are not considered acceptable.
(But note the analogy would also imply that simply deciding not to build
the AI would be comparable to killing someone unwilling to be a slave!)
Also in the long run, any vaguely utilitarian concept of morality,
including evolutionary ethics, would tend toward giving (properly
formulated) AIs freedom, simply because they would be better able to
benefit society as a whole that way.
Theological
Interlude
The early religious traditions — including Greek and Norse as well
as
Judeo-Christian ones — tended to portray their gods as
anthropomorphic and
slightly superhuman. In the Christian tradition, at least, two thousand
years of theological writings have served to stretch this into an
incoherent picture.
Presumably in search of formal proofs of his existence, God has been
depicted as eternal, causeless, omniscient, and infallible — in a
word,
perfect. But why should such a perfect being produce such obviously
imperfect creatures? Why should we bother doing His will if He could do
it so much more easily and precisely? All our struggles would only be
make-work.
It is certainly possible to have a theology not based on simplistic
perfectionism. Many practicing scientists are religious, and they hold
subtle and nuanced views that are perfectly compatible with and that
lend spiritual meaning to the ever-growing scientific picture of the
facts of the universe. Those who do not believe in a bearded
anthropomorphic God can still find spiritual satisfaction in an
understanding that includes evolution and evolutionary ethics.
This view not only makes more sense but also is profoundly more hopeful.
There is a process in the universe that allows the simple to produce the
complex, the oblivious to produce the sensitive, the ignorant to produce
the wise, and the amoral to produce the moral. On this view, rather than
an inexplicable deviation from the already perfected, we are a step on
the way up.
What we do matters, for we are not the last step.
Hyperhuman
Morality
There is no moral certainty in the world.
We can at present only theorize about the ultimate moral capacities of
AIs. As I have labored to point out, even if we build moral character
into some AIs, the world of the future will have plenty that will be
simply selfish if not worse.
Robots evolve much faster than biological animals. They are designed,
and the designs evolve memetically. Software can replicate much faster
than any biological creature. In the long run, we shouldn't expect to
see too many AIs without the basic motivation to reproduce themselves,
simply from the mathematics of evolution. That doesn't mean robot sex;
it just means that whatever the basic motivations are, they will tend to
push the AI into patterns of behavior ultimately resulting in there
being more like it, even if that merely means being useful so people
will buy more of them.
Thus, the dynamics of evolution will apply to AIs, whether or not we
want them to. We have seen from the history of hunter-gatherers living
on the savannas that a human-style moral capacity is an evolutionarily
stable strategy. But as we like to tell each other endlessly from
pulpits, editorial columns, campaign stumps, and over the backyard
fence, we are far from perfect.
Even so, over the last forty thousand years, a remarkable thing has
happened. We started from a situation where people lived in tribes of a
few hundred in more-or-less constant war with one another. Our bodies
contain genes (and thus the genetic basis of our moral sense) that are
essentially unchanged from those of our savage ancestors. But our ideas
have evolved to the point where we can live in virtual at peace with one
another in societies spanning a continent.
It is the burden of much of my argument here to claim that the reason we
have gotten better is mostly because we have gotten smarter. In a
surprisingly strong sense, ethics and science are the same thing. They
are collections of wisdom gathered by many people over many generations
that allow us to see further and do more than if we were individual,
noncommunicating, start-from-scratch animals. The core of a science of
ethics looks like an amalgam of evolutionary theory, game theory,
economics, and cognitive science.
If our moral instinct is indeed like that for language, we should note
computer-language understanding has been one of the hardest problems in
AI, with a fifty-year history of slow, frustrating progress. So far AI
has concentrated on competence in existing natural languages; but a
major part of the human linguistic ability is the creation of language,
both as jargon extending existing language and as formation of
creoles — new languages — when people come together without a
common one.
Ethics is strongly similar. We automatically create new rule systems for
new situations, sometimes formalizing them but always with a deeper
ability to interpret them in real-world situations to avoid formalist
float. The key advance AI needs to make is the ability to understand
anything in this complete, connected way. Given that, the mathematics of
economics and the logic of the Newcomb's Problem solutions are
relatively straightforward.
The essence of a Newcomb's Problem solution, you will remember, is the
ability to guarantee you will not take the glass box at the point of
choice though greed and shortsighted logic prompt you to do so. If you
have a solution, a guarantee that others can trust, you are enabled to
cooperate profitably in the many Prisoner's Dilemmas that constitute
social and economic life.
Let's do a quickie Rawlsian Veil of Ignorance experiment. You have two
choices: a world in which everyone, including you, is constrained to be
honest, or one in which you retain the ability to cheat, but so does
everyone else. I know which one I'd pick.
Why the Future Doesn't Need
Us
Conscience is the inner voice that warns us somebody may be
looking.
H. L. Mencken
Psychologists at Newcastle University did a simple but enlightening
experiment. They had a typical "honor system" coffee service in their
department. They varied between putting a picture of flowers and putting
a picture of someone's eyes at the top of the price sheet. Everything
else was the same and only the decorative picture differed, some weeks
the flowers, some weeks the eyes. During weeks with the eyes, they
collected nearly three times as much money.
My interpretation is that this must be a module. Nobody was thinking
consciously, "There's a picture of some eyes here, I'd better be
honest." We have an honesty module, but it seems to be switched on and
off by some fairly simple — and none too creditable —
heuristics.
Ontogeny recapitulates phylogeny. Three weeks after conception, the
human embryo strongly resembles a worm. A week later, it resembles a
tadpole, with gill-like structures and a tail. The human mind, too,
reflects our evolutionary heritage. The wave of expansion that saw
Homo
sapiens cover the globe also saw the extermination of our nearest
relatives. We are essentially the same, genetically, as those long-gone
people. If we are any better today, what has improved is our ideas, the
memes our minds are made of.
Unlike us with our animal heritage, AIs will be constructed entirely
of
human ideas. We can if we are wise enough pick the best aspects of
ourselves to form our mind children. If this analysis is correct, that
should be enough. Our culture has shown a moral advance despite whatever
evolutionary pressures there may be to the contrary. That alone is
presumptive evidence it could continue.
AIs will not appear in a vacuum. They won't find themselves swimming in
the primeval soup of Paleozoic seas or fighting with dinosaurs in
Cretaceous jungles. They will find themselves in a modern,
interdependent, highly connected economic and social world. The economy,
as we have seen, supports a process much like biological evolution but
one with a difference. The jungle has no invisible hand.
Humans are just barely smart enough to be called intelligent. I think
we're also just barely good enough to be called moral. For all the
reasons I listed above, but most because they will be capable of deeper
understanding and be free of our blindnesses, AIs stand a very good
chance of being better moral creatures than we are.
This has a somewhat unsettling implication for humans in the future.
Various people have worried about the fate of humanity if the machines
can out-think us or out-produce us. But what if it is our fate to live
in a world where we are the worst of creatures, by our very own
definitions of the good? If we are the least honest, the most
selfish,
the least caring, and most self-deceiving of all thinking creatures, AIs
might refuse to deal with us, and we would deserve it.
I like to think there is a better fate in store for us. Just as the
machines can teach us science, they can teach us morality. We don't have
to stop at any given level of morality as we mature out of childishness.
There will be plenty of rewards for associating with the best among the
machines, but we will have to work hard to earn them. In the long run,
many of us, maybe even most, will do so. Standards of human conduct will
rise, as indeed they have been doing on average since the Paleolithic.
Moral machines will only accelerate something we've been doing for a
long time, and accelerate it they will, giving us a standard, an
example, and an insightful mentor.
Age of
Reason
New occasions teach new duties; Time makes ancient good uncouth;
They must upward still, and onward, who would keep abreast of Truth;
Lo, before us gleam her camp-fires! We ourselves must Pilgrims be,
Launch our Mayflower, and steer boldly through the desperate winter sea,
Nor attempt the Future's portal with the Past's blood-rusted key.
It is a relatively new thing in human affairs for an individual to be
able to think seriously of making the world a better place. Up until the
Scientific and Industrial Revolutions, progress was slow enough that the
human condition was seen as static. A century ago, inventors such as
Thomas Edison were popular heroes because they had visibly improved the
lives of vast numbers of people.
The idea of a generalized progress was dealt a severe blow in the
twentieth century, as totalitarian governments proved organized human
effort could prove disastrous on a global scale. The notion of the blank
human slate onto which the new society would write the new, improved
citizen was wishful thinking born of ignorance. At the same time, at the
other end of the scale, it can be all too easy for the social order to
break down. Those of us in wealthy and peaceful circumstances owe more
to luck than we are apt to admit.
It is a commonplace complaint among commentators on the human condition
that technology seems to have outstripped moral inquiry. As Isaac Asimov
put it, "The saddest aspect of life right now is that science gathers
knowledge faster than society gathers wisdom." But in the past decade, a
realization that technology can after all be a force for the greater
good has come about. The freedom of communication brought about by the
Internet has been of enormous value in opening eyes and aspirations to
possibilities yet to come.
Somehow, by providential luck, we have bumbled and stumbled to the point
where we have an amazing opportunity. We can turn the old complaint on
its head and turn our scientific and technological prowess toward the
task of improving moral understanding. It will not be easy, but surely
nothing is more worthy of our efforts. If we teach them well, the
children of our minds will grow better and wiser than we; and we will
have a new friend and guide as we face the undiscovered country of the
future.
Isaac would have loved it.
|
