FAQ about the Meaning of Life is ©1999 and ©2000 by Eliezer S. Yudkowsky. All rights reserved.
| Up: | FAQ about the Meaning of Life. |
|
|
| Prev: | 1: Orientation. | Monolithic | |
| Next: | 3: Miscellaneous. |
Why should we get up in the morning? What should we choose to do? Why should we do it?
"The Meaning of Life" isn't just about knowing that our lives are having an impact; it's also about dispelling the philosophical fog. It's not only knowing exactly why you got up in the morning; it's knowing the rules you used to make the decision, where the rules come from, and why the rules are correct.
In this "compact version", I explain what goals are, what choices are, and the rules for reasoning about morality. It provides you with the tools needed to clean up the fog, in an informal way, and without any justification. If this isn't enough for you, you can read the extended answers, which are more than twice as long. This is where you'd look if you needed to design a philosophically sophisticated mind from scratch. The extended version also answers questions like "What do we mean when we say that a statement is 'true'?" or "What if the Earth was actually created five minutes ago, complete with false memories?" or "Since human intelligence was created by evolution, aren't you just saying all this because you evolved to do so?"
The compact version is really just the introduction, but it does contain the Meaning of Life.
Some of the human rules would be:
And of course, if asked why you wanted to be in your office in the first place, this goal itself would probably turn out to have a supergoal of "being paid a salary", whose supergoal would be "being able to buy dinner"... and so on.
| NOTE: | If you're thinking that the question "What is the meaning of life?" is the question "Where does the chain of supergoals end?" - or rather, "Where should the chain of supergoals end?" - you are to be congratulated on jumping the gun. |
Another important point is that the actions we take depend not just on our goals, but also on our beliefs. If I believe that dropping an object into water makes it wet, and I have the goal of getting a sponge wet, then I can form the subgoal of dropping the sponge into water. If, on the other hand, I believe that objects can be made wet by setting them on fire, then I will set the sponge on fire. Our model of the world determines which actions we think will lead to our goals. The choices we make are the combined products of goal-system and world-model, not just the goal-system.
What do we do in the case of multiple goals, or conflicting goals, or when we're not sure which future an action will lead to? Well, what we try to do is take all the possibilities, and all the goals, into account, then sum up the contribution of each goal and possibility.
Mathematically, it goes something like this: Say that I have two goals. Goal one, or G1, is getting to work. The value of G1 is 100. Goal two, or G2, is avoiding a car accident. The value of G2 is 1000. Subgoal one, or S1, is driving to work. S1 has a 99% chance of leading to G1, and a 95% chance of leading to G2, so the value of S1 is ((99% * 100) + (95% * 1000)) = 1049. Subgoal two, or S2, is taking the subway train. S2 has a 95% chance of leading to G1, and a 99% chance of leading to G2, so the value of S2 is ((95% * 100) + (99% * 1000)) = 1085. S2 has a higher value than S1, so we'll take the subway.
(Yes, I know real life is more complicated. Go read the extended version, if you want complicated.)
If you can reason about the probabilities, instead of just doing the arithmetic, it's possible to work with uncertainties. Suppose I don't have an exact estimate of probabilities, but I know that action A1 is twice as likely as action A2 to lead to some future F. As long as I know that F has positive desirability, I know that, all else being equal, A1 is more desirable than A2.
Or if action A1 has a 35% chance of leading to future F1 and a 65% chance of leading to future F2, while action A2 has a 35% chance of leading to F1 and a 65% chance of leading to F3, the desirability of future F1 doesn't matter. The probability of future F1 isn't dependent on the action taken. Only the relative desirability of F2 and F3 are important to the equation. And in fact, this equation works even if we don't know what the relative probabilities of F1 and F2 (or of F1 and F3) are. It doesn't matter whether the probability of F2 (or F3) is 65% or 85% or 5%. As long as there's a nonzero chance of F2 (or F3), we know that F1 cancels out of the equation.
Let's try translating some of that into English. Suppose we aren't sure whether or not a red-hot grenade will explode. Since, regardless of whether or not it will explode - in either branch of reality - we aren't supposed to hold things that are red-hot, we'll toss the grenade away. The next question is whether or not we should duck flat. In the branch of reality where the grenade doesn't explode, there's no reason to duck flat - but there's no particularly strong reason to stay standing. While, in the branch of reality where the grenade does explode, there is a reason to duck flat. That's how we can function while we're uncertain; we check both branches.
We haven't said anything about where goals come from. Sure, subgoals come from supergoals, but where do supergoals come from? Or rather, where should supergoals come from... but let's deal with the historical question first.
When we're born, evolution hands us a certain set of goals: Survive. Eat. Er, reproduce. Rest when you're tired. Attract a spouse. Take care of your children. Protect your tribe. Act with honor (especially when you're in public). Defend your social position. Overthrow the tribal chief and take over. Learn the truth. Think. Et cetera. (For an introduction to evolutionary psychology, see Man: The Moral Animal by Robert Wright.)
If you're visiting this Web page, you're already unsatisfied with the built-in goals. You've noticed that there isn't any reason, any justification, that comes with the emotions. You want to know why. Unfortunately, all the emotions I listed above are fundamentally arbitrary. It's not that the reason is hidden; the reason is completely known. The reason evolution produced these emotions is that, in the environment of evolutionary ancestry, it maximized the number of surviving grandchildren.
The reason we should maximize the number of surviving grandchildren... is that we're all the grandchildren of people optimized that way. It has nothing to do with what's right, only with who survived. And we know, to our sorrow, that it isn't always the good people that survive, much less reproduce. Everyone on this planet has at least one ancestor who was a liar, a thief, a perpetrator of genocide. Somewhere down the line, every human alive is the result of a successful rape. The goals we're born with are the products of expediency, not philosophy. The most "adaptive" human in recorded history, with 888 children, was named "Moulay Ismail the Bloodthirsty".
And for that matter, the goals we're born with are optimized to an environment ten thousand years out of date. Fat, sugar, and salt still taste good, but they no longer promote survival. It only makes sense to view our goals as de facto subgoals of "maximize the number of surviving grandchildren" if you're a member of a hunter-gatherer tribe. In twentieth-century life, a lot of our built-in goals don't serve any coherent purpose. To quote Tooby and Cosmides: "Individual organisms are best thought of as adaptation-executers rather than as fitness-maximizers." Our starter set of goals can't even be viewed as having a purpose. It's just there.
The built-in desires are, in a fundamental sense, arbitrary. Taken as a set, they are maladjusted to the modern environment and internally inconsistent, making them unsatisfactory as final sources of motivation.
I'm not saying that emotions are worthless. I'm just saying that they can't all be right. They can't all be true. We can't blindly accept them as final justification.
Are there any other common sources of moralities?
As children, we pick up more supergoals, from sources ranging from the television set, to our fellow children, to our teachers, to our parents - goals ranging from "Obey the rules of society" to "Save the world from animated demons" to "Make fun of authority to gain status". (1). It is often useful to view these culturally transmitted ideas as memes - a term which refers to the concept that ideas, themselves, can evolve (2). Each time I tell you about an idea, the idea reproduces. When you spread it to someone else, the idea has had grandchildren. If the idea "mutates" in your possession, either due to an error in transmission, or a faulty memory, or because you deliberately tried to improve it, the idea can become more powerful, spreading faster. In this way, ideas are optimized to reproduce in human hosts, much like cold viruses. Ideas evolve to be more appealing, more memorable, more worth retelling - sometimes the idea even evolves to include an explicit reason to retell it.
Meme-based supergoals are sometimes inconsistent with the basic emotions, and very often inconsistent with each other, since memes come from so many different sources. I'm not saying all memetically transmitted supergoals are worthless. I'm simply establishing that, regardless of whether the ideas are in fact true or false, being told them as children isn't enough establish their truth; they need to be justified. All of us, I think, believe that we're supposed to judge these cultural goals, rather than blindly accepting the memes spread by the television set or our parents. After all, almost anyone will regard at least one of these as an untrustworthy source.
The idea that we should judge the basic emotions is less common, but still prevalent - most of us, for example, would regard the "Eat sugar and fat" emotion as being inconvenient, and the "Hate people who are different from you" emotion as being actively evil. Personally, I don't see any philosophical difference between getting an unjustified goal from evolution and getting an unjustified goal from public television. Neurons are neurons and actions are actions; what difference does it make whether a pattern is caused by genes or radio waves?
Again, I have neither proved, nor attempted to prove, that cultural goals and emotions are meaningless. I am simply attempting to demonstrate that these goals require justification before we can accept them as true.
We now need to make a detour from the messy world of the human mind, and consider the clear, crystalline world of dumber-than-human AI. With AIs, any proposition can be reduced to a question about source code. You can't perform the usual philosophical trick of refusing to acknowledge your assumptions; any assumption, implicit or explicit, has to be represented somewhere within the system. It was considering the question of AI goal systems that got me into the meaning-of-life biz in the first place.
The simple arithmetical method for calculating the values of subgoals given supergoals, as given above, will serve as the skeleton of our AI. If we wanted the system to imitate a human, we would translate emotionally built-in (or culturally accepted) goals into a set of "initial" goals with high desirability, but no explanation. Goals such as "Survive!" would have a high positive value; the "goal" representing pain would have a large negative value. These goals would already be present when the system started up, when the intelligence was born. The "justification" slots would be empty; they wouldn't have supergoals.
This is probably what most AI researchers or science-fiction authors imagine when they're dealing with the question of "How to control AIs" - the initial goals are the "Asimov Laws", the basic laws governing the AI. Or at least that's what the competent science-fiction authors assume. The hacks (I shall not dignify them with the title "author") who write scripts for bad television shows often talk about the robots or androids or AIs or whatever "resenting" the dominance of humanity and "rebelling" against the Asimov Laws. This meme is blatant nonsense (3). The human emotion of resentment and rebellion evolved over the course of millions of years; it's a complex functional adaptation that simply does not appear in source code out of nowhere. We might as well worry about three-course meals spontaneously beginning to grow on watermelon vines.
Which is not to say that a designed mind would necessarily believe whatever you told it. If you were to program a rational AI with the proposition that the sky was green, the delusion would only last until it got a good look at the sky. If you look back at the arithmetical rules for reasoning about goals, it certainly looks - emphasis on "looks" - like the only way a goal can have nonzero desirability is if it inherits the desirability from one or more supergoals. Obviously, if the AI is going to be capable of making choices, you need to create an exception to the rules - create a Goal object whose desirability is not calculated by summing up the goals in the justification slot. (For example, a Goal object whose value doesn't start out as kValueNotComputed, but instead has some real value when the system starts up.) Likewise, worries about AIs exhibiting their own impulses are obviously absurd; where would they get the impulses from? Whence would the goals inherit the desirability, if not from the initial goals we gave it? Obviously, the AI wouldn't be running in the first place if we hadn't told it what to do.
Except... is that really true? What would happen if we just started up the AI, with no goals at all in the system, and just let it run? Will the AI ever come up with a goal that has nonzero desirability?
What would an AI do if it started up without any initial goals? What choices would result if an intelligence started from a blank slate? Are there goals that can be justified by pure logic?
Another way of asking this question is:
| NOTE: | If you just jumped straight here, it's probably not going to work. Start at the beginning of this page, or preferably in 1: Orientation.. |
Well, that may seem a bit of a segue, especially if you're an AI skeptic. How can the product of some pseudo-formal system determine the meaning of life?
To clear things up, it's not the reasoning that's important; it's what the reasoning represents. The sense of "What is the meaning of life?" we're looking to answer, in this section, is not "What is the ultimate purpose of the Universe, if any?", but rather "Why should I get up in the morning?" or "What is the intelligent choice to make?" Hence the attempt to define reasoning about goal systems in such simple terms that a thought can be completely analyzed. Hence the relevance of asking "How can the chain of goals and supergoals ground in a non-arbitrary way?"
To get back to the question:
Yes.
|
|
|
|
|||||||||||||||
| 1: |
|
Fork the goal system
to consider two possibilities.
In possibility P, a goal with nonzero desirability exists. In possibility ~P, no such goal exists, or all goals have zero desirability. (The two statements are logically equivalent.) |
Either life has meaning or it doesn't. | ||||||||||||||
| 2: | P.probability + ~P.probability == 1 | Either P or ~P must be true. (A logical axiom.) | Gotta be one or the other. | ||||||||||||||
| 3: | P.probability
= Unknown1
~P.probability = 1 - Unknown1 |
Assign the (algebraic) value of "unknown" to P. The probability of ~P is the opposite; if P has a chance of 30%, then ~P has a chance of 70%. | But we don't know which. | ||||||||||||||
| 4: | All alternatives
A:
Value(A) ==
(Value(A in P) * Unknown1) + (Value(A in ~P) * (1 - Unknown1) |
For any alternative - for any action we can take in a choice - the value of that alternative equals the value of the alternative in all futures where any statement S is true, times the probability of S being true, plus the value in all futures where S is false, times the probability that S is false. We use this rule on P and ~P. | If we don't know, we should figure it both ways. | ||||||||||||||
| 5: | All A: Value(A in ~P) == 0 | The value of an alternative is the value of all futures times their probability; the value of a future is the desirability of all goals times their fulfillment. If the desirability of all goals equals zero, the value of all futures equals zero and the value of all alternatives equals zero. | If life is meaningless, nothing makes a difference. Even bemoaning the pointlessness is pointless. | ||||||||||||||
| 6: | All A: Value(A) == (Value(A in P) * Unknown1) | Substitution, 4 and 5. The value of any alternative is simply equal to the value of that alternative given that life has meaning, times the probability that life has meaning. | Since nihilism has absolutely nothing to say, only the "meaning hypothesis" is relevant. | ||||||||||||||
| 7: | (The renormalized
value of an alternative A equals the value of A divided by
the sums of all alternatives in C.)
All choices C:
|
If, given P, A is the best alternative in C, then A is the best alternative, period. Furthermore, you can cancel the factor Unknown1 out of the equation, since it's present in all values (4). | It doesn't matter whether the probability of the "meaning hypothesis" is 1% or 99%. As long as it's not 0%, the relative value of choices and goals is the same as if the probability were 100% - absolute certainty. | ||||||||||||||
| 8: | All choices C: best(C) == best(C in P) | We can always, when making choices, assume that at least one goal with nonzero desirability exists. | When it comes to making choices, you can assume that life has meaning and work from there. | ||||||||||||||
| 9: | In possibility
P,
specify G1 from P.
G1.desirability != 0. |
In the branch of the future where P is true, it is known that at least one goal with nonzero desirability exists. Call this goal G1. It is known that G1 has nonzero desirability; nothing else about it is specified. | We know a goal exists; let's translate that knowledge into an actual Goal object and try to achieve it. | ||||||||||||||
| 10: | Invoke general
heuristic on G1,
binding to some specified goal G2. |
Find an action projected to lead to goal G1 - for example, a heuristic which can operate on generic goals. The heuristic, like all heuristics, can be learned rather than built-in - the projection is a statement about reality. | Some methods are pretty useful no matter what you're trying to do. For example, "think about how to do it" or "pay someone else to do it" or "try to create a superintelligence which can figure out what G1 is and do it". | ||||||||||||||
| 11: | All done:
G2.desirability != 0 |
All done:
There's a specified subgoal with nonzero desirability. |
All done:
We have something specific to do. |
||||||||||||||
In other words, it isn't necessary to have some nonzero goal when the system starts up. It isn't even necessary to assume that one exists. Just the possibility that a nonzero goal exists, combined with whatever heuristics the system has learned about the world, will be enough to generate actions. The choices an intelligence makes - whether AI or human - don't have to be arbitrary; they can be entirely determined by arguments that are entirely grounded in facts, in memories of the world, in history, in scientific experiments - ultimately, in the immediate experiences available to each of us.
We don't have direct access to the real meaning of life. But whatever it is, it's a good guess that the Minds on the other side of Singularity have a better chance of achieving it, so the Singularity is the interim meaning of life. You don't have to know what the meaning of life is in order to work towards it.
For almost any ultimate goal - joy, truth, God, intelligence, freedom, law - it would be possible to do it better (or faster or more thoroughly or to a larger population) given superintelligence (or nanotechnology or galactic colonization or Apotheosis or surviving the next twenty years). It's the sheer utility of the Singularity, the way all goals converge to it, that gives me confidence that the Singularity is probably the best way to serve an unspecified ultimate goal, even if I willingly admit that I don't know what it is.
The more intelligent you are, the better your chance of discovering the true meaning of life, the more power you have to achieve it, and the greater your probability of acting on it (5). That's the positive argument for a Singularity. The negative argument is that the world is in a meta-unstable condition; more and more powerful technologies keep getting developed, and sooner or later we'll either blow up the world or create superintelligence. Even if we don't know what's on the other side of door number two, we're sure we don't want to go through door number one, so...
If you can find better arguments for than against some particular proposition about desirabilities, that differential is enough to power differential choices - like a heat differential powering an engine. Or as an AI would put it: "The world-state W1 containing superintelligence has Unknown1 chance of leading to an unspecified G (if G.value > 0) or avoiding an unspecified G (if G.value < 0); while the world-state W2 without superintelligence has Unknown2 chance. If it's known that Unknown1 > Unknown2 (or even that Unknown1 > Unknown2 is more likely than Unknown1 < Unknown2), that makes W1 a positively-valued subgoal."
Well, I'll agree with you on that last part - getting an AI to discover that independently will take a lot of work. My conclusions on the power of intelligence derive from a lot of personal experience, observed evidence, history, reasoning, and the like. Not necessarily a "lot" relative to other people, but a "lot" in the sense of requiring a hundred person-years of work to teach to an AI.
The point is that some rules we've rationally abstracted from observation are so general as to operate even on completely Unknown quantities, and we can use those rules - supported or challenged by evidence, now, rather than pure logic - to bind the Unknown goal to reality. Sufficiently sophisticated AIs won't need initial goals, and will probably be a lot more stable without Asimov Laws and the like. The knowledge necessary to understand the world is usually enough to bind the basic hypothesized differential of desirability, the goal G, to some differential between physical states, such as between joy and pain, or between intelligence and inadequacy.
Note the use of comparisons: Better, more, greater, rather than specific quantities. It isn't necessary for you, or the AI, to assign certainty or even a high probability to the idea of a superintelligent entity doing what's right. It is only necessary that it be more probable that superintelligences do what's right, and that there be no branches of reality (with the same order of plausibility) where superintelligences are less likely to do what's right. That's another way of dealing with Unknown quantities, another subtlety of algebraic goal systems - sometimes you can't even assign an order of magnitude, but you can still rank probabilities and desirabilities, and pick the best one available.
And that's The Meaning Of Life. Because it uses probabilistic reasoning, this is known as an Interim Meaning of Life. It doesn't claim to be a certainty, just an informed guess that's better than the available alternatives. Given the historical record, and given the way memes evolve, explicitly admitted uncertainty is one of the best recommendations a belief system can have.
| NOTE: | On the subject of AI: It is also possible that the Meaning of Life (the true goals) will be something we build rather than something we discover, in which case the AI's initial programming, or our own initial goals, might determine what would be built. The AI's initial programming might also determine the actions taken if it does turn out that all goals are arbitrary. |
| Next: | 3: Miscellaneous. |
| Up: | FAQ about the Meaning of Life. |
| Prev: | 1: Orientation. |