Cooperate without looking: Why we care what people think and not just what they do Moshe Hoffman', Erez Yoeli', and Martin A. Nowak' I' •Peogram for Evolutionary Dynamics, Oeparunent of Mathematics and 'Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, MA 02138 Edited by Michael S. Gananiga, University of California, Santa Barbara, CA, and approved December 24. 2014 (received for review September 30, 2014) Evolutionary game theory typically focuses on actions but ignores motives. Here, we introduce a model that takes into account the motive behind the action. A crucial question is why do we trust people more who cooperate without calculating the cost? We propose a game theory model to explain this phenomenon. One player has the option to 'look" at the costs of cooperation, and the other player chooses whether to continue the interaction. If it is occasionally very costly for player 1 to cooperate, but defection is harmful for player 2, then cooperation without looking is a subgame perfect equilibrium. This behavior also emerges in population-based processes of learning or evolution. Our theory illuminates a number of key phenomena of human interactions: authentic altruism, why people cooperate intuitively, one-shot cooperation, why friends do not keep trade of favors, why we admire principled people, Kant's second formulation of the Categorical Imperative, taboos, and love. game theory I evolution I emotion I motive I cooperation C ooperation OMITS when we lake on costs to help others. A key mechanism by which cooperation is sustained is reciprocity: Individuals cooperate with those who have cooperated in the past (1-14). However, we care about not only whether others cooperate but also. their decision-making process: wc place more trust in cooperators who do not strategically weigh the arts and make an effort to collect than before deciding whether to cooperate. For example. we are impressed by colleagues who immediately agree to proofread a paper but view with suspicion those who ask, "how many pages does it have?" Intuitively. those who cooperate without "looking" (CWOL) can be trusted to cooperate even in times when there arc large temptations to defect. However. will the added trust from CWOL be worth missing out on those large temptations? Additionally, which conditions make CWOL a winning strategy? To address these questions. we develop thc envelope game (Fig. I). which is a repeated asymmetric game between two players. In each round. player 1 receives an envelope. which contains the magnitude of the temptation to defect. The temptation is low with probability p and high with probability 1 —p. Player 1 can choose to look inside the envelope and, thus, find out the magnitude of the temptation. Then. player 1 decides to cooperate or defect Sub- sequently. player 2 can either continue or end the game. In the former case there is another round with probability w. If player I cooperates. her payoff is a. whereas player 2 receives b. If player I defects. her payoff is either cr or ca, depending on whether the temptation is low or high, respectively, whereas player 2 receivesd. We have thc following inequalities: ca > q> a> 0 and b>0>d. Moreover..we have pb + (1 —p)d O. Therefore. player 2 prefers not to interact with a player 1 who only cooperates when the temptation is low. Finally. wc assume that low temptation is more likely than high temptation: p> 1/2. To understand thc essence of the game. we need to consider four strategics for player 1 and three strategies for player 2. The player 1 strategies am (i) CWOL, (ii) cooperate with looking (CWL), (iii) look and cooperate only when the temptation is low. and (iv) always defect. The player 2 strategics are (i) end if player 1 looks, (ii) end if player 1 defects, and (iii) always end. In SI Appendix, wc also explore a richer strategy set. The payoff matrix is shown in Table I. The strategy pair "always defect" and "alums end" (ALLD) is always a Nash equilibrium: no player can increase her payoff by deviating unilaterally. However, there are other Nash equilibria. All proofs are in SI Appendix. Of particular interest is the strategy pair where player 1 chooses CWOL and player 2 aids the game if player 1 looks. This strategy pair is a Nash equilibrium ifa/(1 — w)a.cip +ch(1 — p). This condition has a natural interpretation: player I's expected temptation from de- fection is less than the gains from an ongoing cooperative in- teraction. The expected temptation matters because, if player 1 were to look, player 2 would end the relationship. Thus, player 1 might as well defect. regardless of the temptation. Not looking, in a sense, smooths the temptation to defect; the variability in temptations no longer matters. Another relevant strategy pair is if player 1 CWL and player 2 ends if player 1 defects. This pair is a Nash equilibrium if a/(1 — w) ≥ ch. This condition has the following interpretation: to sustain CWL, the long-term gains to player 1 from the ongoing relationship must suffice for player 1 tocooperate. even if player 1 knows the temptation is high in the current period. For CWL. it is the maximal temptation that matters: because player 1 is not pe- nalized for looking. she can look at the temptation and choose to defect only if it is high. When it is oa:asionally very costly to cooperate kip +ch(1 — p)≤ a 1(1 —w)<chl, CWL is not an equilibrium. but CWOL is. This expression identifies the region where we should be most likely to discover CWOL In SI Appendix, we show that the inequality pb+ (1 — p)d <0 must hold for CWOL to emerge. In SI Appendix. we also address some concerns. First, after wc consider a richer strategy set. there might be other equilibria of the envelope game, in which player I sometimes does not look. These equilibria might exist undcr different conditions from those given above, which may draw into question our statement that looking matters under these conditions. We show that, whcn we rule out strategies that randomize or depend on the round, Significance Why do we trust people more when they do good without considering in detail the cost to themselves? People who avoid "looking" at the costs of good acts can be trusted to cooperate in important situations, whereas those who look cannot We find that evolutionary dynamics can lead to cooperation without looking at costs. Our results illuminate why we attend closely to people's motivations for doing good, as prescribed by deontological ethicists such as Kant, and, also, why we admire principled people, adhere to taboos, and fall in love. Author tontaouticm M.N, E.Y., end IA A N designed restock performed research and wear the paper. The mothers dedere no conflict of interest Thiswild, is e PNAS 0irect Submission. Freely wallal. °name bough the PNAS open aaess opton. 'To wheal correspondence Avid he addressed ETNI: maronficwakeharvard.odu. This article conteire 'updating information online e wAw gine of 9gooktokupe1/4104 10. iofirpos.1417904112/4/Gupplemostat vawa.pnas.orgicgirdeirl 0.1073/pnas.1417904112 PNAS Early Edition I 1 of 6 EFTA00632631

Temptation to Defect Low High (1) 'e4 (2) (3) (4) Fig. 1. The envelope game. Cl) The game begins when the temptation to defect is randomly chosen, as indicated by a notice being placed in the envelope. The temptation to defect is low with probability p and high with probability 1 - p. (2) Player 1 receives the envelope and chooses whether to look (open the envelope). (3) Player I decides whether to cooperate or defect Player 1 can only condition her action on the realized temptation if she has looked. Each time that player 1 cooperates, regardless of whether player I looked, both players benefit from the interaction: player 1 gets a > 0, and player 2 gets b> 0. Player 1 gains even more if she defects. If the temptation is low, player I gets q> a, and if it is high, player 1 gets ca >q. In either case, each time that player I defects, player 2 is harmed and gets a negative payoff (cl <0). Moreover, we assume that the harm is substantial Cal c -bp1(1 -p11, and therefore, player 2 prefers not to interact with a player 1 who only cooperates when the temptation is low. (4) Player Z having observed both of player l's choices, decides whether to continue or end. If player 2 continues, there is another round with probability w. ALLD, CWOL, and CWL are the only equilibria of the envelope game. even with this richer strategy set. Second, because player 2 does not directly benefit by attending to looking, she might not do so. This concern proves moot. The intuition is that if there is even a small probability that player 1 looks. player 2 is better off attending to looking. We formalize this intuition by showing that this equilibrium, as well as ALLD and CWL. can be made subgame-perfect, which is a solution concept used to rule out these kinds of concerns (15). In many cases we do not consciously avoid looking or distrust those who look. but. rather, are guided to do so by a gut sense, an emotion, or an ideology. That is. looking feels or is wrong. Where do these emotions and ideologies come from? Individuals do not adopt them rationally or even consciously. Therefore, we now consider the case where strategies (such as. it feels wrong to look) are learned or evolved. We use the replicator dynamic. which is the standard model for evolutionary dynamics (16-18). and also described rein- forcement learning and prestige-biased imitation (19). The rate of reproduction is proportional to the payoff that a strategy receives. Because we have two types of players. our simulation studies coevolutionary dynamics in two populations. Players of type 1 can adopt one of four strategies described above. Players of type 2 can adopt one of three strategies described above. Our state space is the product of the simplex Sy and the simplex Sa. A point in the simplex Sy describes a strategy mix of type 1 players. A point in the simplex 53 describes a strategy mix of type 2 players. We randomly seed the strategy frequencies many times and re- cord the frequency of each strategy after the population has sta- bilized. We observe three possible outcomes that correspond to the Nash equilibria deserted above (Fig. 2). (i) Type I players con- verge to always defect. whereas type 2 players converge to a tri- angular region close to always end. (ii) Type 1 players converge to CWOL whereas type 2 players converge to a mixture between end if player 1 looks and end if player 1 defects. For stability, this mix- ture must contain a minimum fraction of end if player 1 looks. (iii) Type 1 players converge to a mixture between CWOL and CWL, whereas type 2 players converge to end if player 1 defects. The dynamic stability of those evolutionary outcomes coincides with the criteria for the underlying strategy pairs to be Nash equilibria We now apply the model to shed light on some questions di- rectly related to cooperation. First, psychologists and philosophers have long asked the following question: is helping others -always and exclusively Table 1. Payoffs for a restricted set of strategies in the envelope game Player 1 End If Player 1 looks CWOL CWL Look and cooperate only when temptation is low Always defect Player 2 End If Player 1 defects Always end * T Ws a,b 4P+40 - PMN3+d(t - P) co+ chit —Aci Tfre • eV; Ite•41.- 0 412+ bin - PLd a,b ab aP+40 - PhbP+ - P) op+ chil—Acil Player l's strategies are presented in rows, and player Zs strategies are presented in columns. The payoffs at the intersection of a given row and column ate those that the players receive if they play the corresponding strategies. For example, if player 1 looks and cooperates only if the temptation is love and player 2 ends if player 1 defects, then player l's expected payoff is lap+ Ca (I -p)j/il - pwl, and player 2's expected payoff is lisp+ 4(1 -pjj, - pry . Details of regulations leading to payoffs are in SI Appendix. DepencIng on the parameter values, there are up to three Nash equilibria. The pair (ALLD) is always a Nash equilibrium. The pair (CWOL and end if player 1 looks) is a Nash equilibrium if a/(1 -w)> qp+ ch(1 -p). The pair (CWL and end if player I defects) is a Nash equilibrium if .1(1- ta)> q,. We refer to these strategy pairs as ALIO, CWOL, and CWL, respectively. 'Nash equilibria of the envelope game. 2 of 6 www.was.wgicgifd:010.1073/was.1417604112 Hoffman et aL EFTA00632632

Equilibrium Classification Player 1 Player 2 All D: CWOL CWOL: T. t, CWL: 00 1.0 o.e cg 0.6 E 0.4 U. 0.2 00 1.0 g 0.8 a) 0.6 a) 0' 0.4 LL 0.2 00 041-w) (per • (1-Nol(14.) Imalk " I PL oak Payoff a Fig. 2. Learning dynamics of the envelope game. We randomly seed the strategy frequencies 10,000 times for 50 values of the payoff value a and record the frequency of each strategy after 1,000 generations. We observe three possible outcomes that correspond to the Nash equiltela identified in Table I. (i) Type I players converge to always defect, whereas type 2 players converge to a triangular region close to always end. Cu) Type 1 players converge to CWOL, whereas type 2 players converge to a mixture between end if player I looks and end if player 1 defects. For stability, this mixture must contain a minimum fraction of end if player I looks. Oil) Type I players converge to a mixture between CWOL and CWI., whereas type 2 players converge to end if player 1 defects. We vary the value of a along the x axis. They axis represents frequencies., and each colored line presents the frequency of each outcome. The parameter region where the corresponding strategy pair is supported as an equilibrium is shaded. Additional detaib are in Si Appendix. All D, all defect; C, converge. motivated by the prospect of some benefit for ourselves, however subtle" (20) [for example, the conscious anticipation of feeling good (21), avoidance of guilt (22-24). reputational benefits, or reciprocity (1-14)j. At the extreme, this question amounts to ask- ing if saintly individuals, such as Gandhi or Mother Teresa. were motivated thus or if they were authentic altruists who did good without anticipating any reward and would be altruistic. even in the absence of such rewards. Our model suggests that authentic altruism is. indeed. possible: by focusing entirely on the benefits to others, authentic altruists are trusted more, and the benefits from this trust outweigh the risk of. for example. dying a martyr's death. Second, we address another question of why people are intuitive cooperators. That is when people decide rapidly. they are more likely to cooperate than if they have time to deliberate. Addition- ally, people who cooperate decide more quickly than those who defect (25-27). Intuitive cooperation underlies extreme acts of heroism, acts which, because they place the hero at great personal risk, are otherwise difficult to explain (2S). The Social Heuristics Hypothesis offers one explanation for this phenomenon: we adopt heuristics to avoid incurring cognitive costs associated with de- liberation (29-31). In a world with repeated interactions, it is usually worthwhile to cooperate. and therefore, individuals may adopt heuristics. such as always cooperate or always cooperate in nonbusiness settings. These same individuals, when serving as laboratory subjects. may apply these heuristics and cooperate even when it is not worthwhile to do so (25, 32.33). Our model offers an additional explanation for intuitive co- operation: intuitive cooperation may serve to reduce respon- siveness to costs of cooperating in a particular situation. For this explanation to be sensible, it must be the case that whether a decision is made intuitively or deliberately is detectable. In fact, it is: deliberative decision-making leads to slower reaction time as well as increased pupil size and heart rate (34) and sometimes, blushing or stammering (35). Our model makes two predictions that one would not make if the Social Heuristics Hypothesis. by itself, explained intuitive cooperation: decisions related to cooperation are more likely to be intuitive than other decisions that are similarly usually worthwhile, and intuitive Hoffman et al. PNAS Early Edition 1 3 of 6 EFTA00632633

cooperators are trusted more than reflective cooperators. There is evidence for the latter: in an experiment eliciting moral judgments. subjects who read vignettes about people who re- turned lost wallets judged those who returned the wallets without hesitation more positively than those who hesitated (36). Third. we address the question of why people cooperate in one- shot situations (for example, in laboratory experiments, such as the dictator game) (37). Cooperation in these situations is waling from the standpoint of models of the evolution of cooperation based on reciprocity. because in that framework. cooperation can only im- prove one's reputation if actions are observed, and reputations are only valuable if others have an opportunity to reciprocate. Some have suggested that cooperation in these settings results from intuitive cooperation (25, 32), that it can emerge if there is un- certainty over the probability of future cooperation opportunities (38). or that it is a consequence of the evolution of altruism caused by group selection (37). We offer another potential ex- planation by considering a variation of our model, in which the likelihood of continuation varies and player 1 learns this likeli- hood when she looks (details in SI Appendix). This model sug- gests that subjects learned or evolved to not consider who is watching so that others can expect them to cooperate. even when no one is. Unlike the other explanations cited, our explanation implies that, when there ends up being an opportunity to interact again, those who cooperate when they thought no one could reciprocate will be rewarded more than those who cooperated knowing someone could reciprocate. Indeed, laboratory subjects cooperate more with those who cooperated with a third party under the presumption that no one would have a chance to re- ciprocate (39). Moreover, our explanation uniquely predicts that people would feel wrong attending because of the fact that the situation is one shot and that others will judge them harshly if they behave differently when the situation is one shot. Fourth, we address the question of why we find it unbecoming when dose friends keep track of favors or reciprocate favors im- mediately. In experiments, subjects ) do not cooperate more with friends who have just given them a gift but do cooperate more with strangers who have just given them the same gift (40). (ii) take greater care to highlight contributions to strangers than to friends (41). (iii) are offended when close friends immediately reciprocate kind acts but not when strangers do (42). and (iv) judge friendships as less close when those relationships display immediate reciprocity (43. 44). In fact. relationships have been shown to fall into distinct categories. in part characterized by whether favors are tracked (45). These observations have led researchers to conclude that - the dynamic of friendship does not fit the logic of models of reciprocity and presents a (mini,- for evolutionary• analysis" (43). However, our model suggests an explanation that is consistent with reciprocity. If close friends CWOL their decision to cooperate is affected not by a single. recent kind act but rather. only by the distribution of payoffs from the relationship in the long run. Moreover, when the beneficiary• of a good deed immediately reciprocates. then either the beneficiary is looking or the beneficiary thinks that the friend who did the good deed was looking. Fifth. our model gives insight on a number of interesting phenomena not prima facie related to cooperation. Why do we like people who are principled and not like those who are strategic? For example. we trust candidates for political office whose policies are the result of their convictions and consistent over time, and we distrust those whose policies arc carefully constructed in consultation with their pollsters and who flip-flop in response to public opinion (as caricatured by the infamous 2004 Republican presidential campaign television ad- vertisement showing John Kerry windsurfing and tacking from one direction to the other). Instead of respecting politicians who flexibly respond to public opinion, we view them as sleazy. Our model offers the following potential explanation. Someone who is strategic considers the costs and benefits to themselves of every• decision and will defect when faced with a large temptation, whereas someone who is guided by principles is less sensitive to the costs and benefits to themselves and thus, less likely to defect. Imagine that our flip-flopping politician was once against gay marriage but supports it now that it is popular. That he only sup- ports it when it is popular indicates that the politician is unlikely to fight for the cause if it later becomes unpopular with constituents or risks losing a big donor. Note that, not only will gay rights activists distrust the flip-Hopper but also. women's rights activists will dis- trust him, even if the flip-flopping politician has always supported women's rights, because the flip-Hopper would be likely to end his support for women's issues if it is ever advantageous for him to do so. Of course, we do want our politicians to be strategic about some things. For example. we would prefer that they carefully consider fatalities before invading a foreign country•. Our model suggests that we would like pulitkians—and others more generally—to be strategic about the costs and benefits to us (fatalities) but not the costs and benefits to themselves (likelihood of getting reelected). Our model also teaches us when we will not be bothered if others are strategic: when defections are either not especially tempting top + ch(1 —p)≤a /I —w] or not especially harmful [bp + d(1 — p}<0[b]. Contrast the flip-flopping politician with a business partner who might have the opportunity to cut you out of your latest deal. As long as such a temptation benefits your partner little relative to losing a valuable long-term partnership, your partner would never be tempted. and you need not be bothered if he is strategic. Next, we discuss why we feel moral disgust by those who use or manipulate others, as famously condemned by Kant in his second formulation of the Categorical Imperative: -Act in such a way that you treat humanity ... never merely as a means to an end, but always at the same time as an end" (46). Consider the well- known example of dwarf-tossing. Many sec it as a violation of dwarves' basic dignity to use them as a means for amusement. although dwarves willingly engage in the activity for economic gain. Our aversion to using people may explain many important aspects of our moral intuitions, such as why we judge torture as worse than imprisonment or punishment. Our model suggests that we are repulsed by those who treat others as a means to an end, because they are liable to mistreat their relationship part- ners when expedient. even if, currently, the relationship is mutually beneficial. The previous two applications are examples of a more general phenomenon: that we judge the moral worth of an action based on the motivation of the actor as argued by deontological ethi- cists but contested by consequentialists. The deontological ar- gument is famously invoked by Kant (46): Action from duty has its moral worth not in the purpose to be attained by it but in the maxim in accordance with which it is decided upon. and therefore does not depend upon the realization of the object of the action but merely upon the principle of volition in ac- cordance with which the action is done without regard for any object of the faculty of desire. These applications illustrate that we attend to motives because they provide valuable information on whether the actor can be trusted to treat others well, even when it is not in her interest. Next, we consider why people dislike considering tradeoffs related to "sacred values" (47). Sacred values are values, such as love. liberty, honor, justice. or life, that people treat -as pos- sessing transcendental significance that precludes comparisons. tradeoffs, or indeed any mingling with secular values•' (47). Al- though there is variation in what societies consider sacred. vir- tually all societies have a concept of sacredness (47). Sacred values are so strongly imbued in us that we do not find them puzzling prima fascia. but their existence and origin remain poorly understood. What makes us treat some values as sacred. 4 of 6 www.ecas.oigkgifdo110.1073/plas. 1417904112 Hoffman et S. EFTA00632634

and what differentiates these values from secular values. like free time or money. that we more readily trade? Our model provides one possible explanation. People who calculate costs of trading off against sacred values are less trustworthy when it comes to safeguarding these values than people who consider them sacred and would never calculate the costs of trading off against them. Responding with disgust to these taboo tradeoffs may be one way to prevent us from inter- acting with people who make such tradeoffs and hence. are less trustworthy and, also. may be a way to signal to others that we ourselves would not consider and therefore, make such tradeoffs. Consistent with CWOL, it is taboo to consider the tradeoff. even if one ultimately makes the right choice. and the longer the tradeoff is considered for, the harsher the judgment by observers (47). Importantly, those who consider a taboo tradeoff, such as selling their own child, pay a reputational cost, because such considerations indicate that one, in general. does not hold sacred values and cannot be trusted with, for example. care of other•' children, the elderly, or shared resources. If CWOL. indeed, underlies the phenomenon of taboo trade- offs, then it provides two predictions. First, taboo tradeoffs will prevail precisely in situations where there is large but infrequent temptation to defect and defection is harmful. such as selling a child, betraying a country. or sleeping with someone for a million dollars. It remains to be shown that taboo tradeoffs show these characteristics. Second, it also provides an important policy prescription regarding policies forbidding taboo tradeoffs (for example. the ban on euthanasia): such policies are socially suboptimal. because the benefits of cooperating without looking accrue to the individuals who advocate them, but the costs are borne by society. We note that the above arguments extend to taboos in general and explain why they often have the property that it is not merely a transgression to violate the taboo but to just consider violating it (48). Finally, our model offers an explanation for emotions, such as love, which is closely related to the explanation first proffered by Frank (49) [precursory insight is given in the work by Schelling (50); also see the works by Hirshleiter (51). Pinker (35). and Winters (52)]. Love has the property that we behave altruistically toward our partners, regardless of what temptations arise (49). as illustrated by the wedding vow - for better or for worse, for richer, for poorer, in sickness and in health.•' For example. love causes individuals to ignore other potential mates, even if those mates are better than one's current mate, as Shakespeare's Juliet did when her love for Romeo led her to rebuff the advances of the otherwise more suitable Paris. Why does love have this property? Our model suggests that those who are blinded by love can be trusted to stay with their partners in sickness and health. because they are not looking at the costs of cooperation in these diverse situations. This expla- nation for love is different from the explanation by Frank (49). Frank (49) argues that those who are blinded by love observably commit to staying with their partners That is. those who are in love today do not have the option to defect tomorrow. The ar- gument by Frank (49) has been criticized because one could I. Wren 81 (1931) The evolution of fanatical altruism. 417 Roy tb.0 8511335-57. 2. Friedman 1W (1971) A non-cooperative equilibrium for woectornm Rev (con Stud 38(1)1-12. 3. Axelrod RM (198,1) The (volution of Cooperation Manic Books. New York). 4. fudenberg D, Maskin E 11986) The folk theorem in repeated games with dscounag or with incomMete information. Econometrka 540)533-554. & fudenberg D. Mask n E ( 990) Evoluton and cooperation m noisy repeated games. Am Leon Rev SOO:274-279. 6. (drew, K4 Sanunbon I. (19921 Evolutionary stability in repeated genies played by finite automata. I Icon Theory 57:278-305. 7. Him* MA. Sigmund K 11992) Ti for let in heterogeneous Populatioiwalarive 355: 250-253. & Hawk PA, Sigmund K (1993) A strategy of win.stay, lose4rft that outperforms lit. for.tat In the Prisoners Dilemma game NOW. 361(64321:56-58. evolve to be in love today and defect tomorrow. Our model requires a different and we think, more realistic constraint: that it is impossible to look while one appears as though not looking. This assumption is justified by the fact that, at least in some contexts, gathering information about the costs and benefits is inherently observable (for example. through reaction time or the questions that one asks). Existing evidence is consistent with both models: emotions related to love are observable (53). cannot be faked (54). and are relied on by partners when choosing whether to cooperate (55. 56). There is also reason to believe that love and related emo- tions would be hard to fake given their autonomic origins and the costs of placing their activation under conscious control (35,49). It remains to be shown that love, in particular, has these attrib- utes and that we cannot evolve or learn to display love while still attending to costs, Thus, additional research is warranted to differentiate between the model by Frank (49) and CWOL. Consistent with CWOL, mere discussions of the costs and benefits of a relationship or a breakup (for example, suggesting a prenuptial agreement) damage the relationship. Such dis- cussions indicate that one is looking at the costs of the re- lationship and cast doubt on one's commitment. CWOL also elucidates that falling in or out of love depends on the distribu- tion of temptations but not their immediate realizations, sug- gesting that people will fall out of love when there is a permanent change in alternative mating opportunities or relationship casts but not when there is a one-off temptation. For example. one may fall out of love with one's partner after becoming unexpectedly successful. Finally, CWOL clarifies that love comes with a cost— the cost of ignored temptations—and suggests that this cost must be compensated for with commensurate investment in the re- lationship. Only sometimes is it worthwhile for the recipient of love to compensate a suitor. which explains why people actively avoid the strung affections of those with whom they do not wish to have long-term relationships. These arguments extend to anger. Anger can be thought of as -punishing without looking.•' It prevents people from looking at the costs of inflicting harm on others after a transgression. thereby deterring future transgressions. This paper formalizes a simple intuition first spelled out by Trivers (I): One can imagine. for example. compensating for a misdeed without any emotional basis but with a calculating. self-serving motive. Such an individual should be distrusted because the calculating spirit that leads this subtle cheater now to compensate may in the future lead him to cheat when circumstances seem more advantageous (because of unlikelihood of detection, for example. or because the cheated individual is unlikely to survive). We hope that formalizing this intuition has added valuable in- sight on otherwise puzzling aspects of human nature. ACKNOWLEDGMENTS. This research was funded, in part, by John Templeton Foundation Grant RFP-12-11 from the Foundational Questions in Evolution- ary Biology Fund, National Science Foundation Grant 0905645, and Army Research Office Grant W91 INF-11-1-0363. 9. Aumann ID. shame, LS 119941 long-Term Companion an Game.Thcatetk Analph (springer. Berleil. 10. Now* IAN Sigmund K (199() Evolution of incited reogrotity be image scoring Nita 393(6685):573-577. 11. Nowak MA. Sigmund K (20051 Evolution of inbred retipraily. Neese 43717063): 1291-12% 12. Nowak MA 0006) Pa nies for the evolution of cooperation. Science 314158058 1560-1563. 13. Ohtsulr.I N, Mesa V (2006) The leading eight: Sodal norms that can maintain co. operation by indirect reciprocity. 1 MOP gio)239141435-444. 14. Sigmund K 11010) The Crittuko of Serfishnen (Princeton Univ Press. Princeton/. IS. Osborne M1(20031 An Infioduction to Come Meow Oxford link. Press• Hew TON. I& Hofbauer 1, Sigmund K (1998) Evolutionary Games and PopuAnlon Oynamin (Cambridge Univ Press, Cambridge, United Kingdom). I ce Hoffman et al. PAS Early Edda, I S of 6 EFTA00632635

17. Weibul SW (1997) fmkeionary Game Theory DAR Press, Cambridge. MN. IL Nowak MA 120061 Evolvtionary Lynam.= &profile the Equations of Lob (Harvard UMW Press. Cambridge. MA). 19. Fudenberg DA (1996) The Theory of Learning in Gamer (MIT Press. Cambridge. MAL Vol 2. 20. Batson CD (2014) The Aftnai.vn Question: reword s Sociol.esychologioif Answer (Psychology Press. Hew York/ 21. And eon 1( 990) trope e altruism and donations to public mice A theory of warm. glow giving (con ) (London) 10014011464-477. 22. Dana 1, Cain DM, Dawes MA (2006) What you don't know wont hurt me: Costly (but quiet) exit in dictator genres. Organ Sehev Num Danis Process 100193-201. 23. Dellelligne S. List /A. Melinender U 42012) Testing for statism end social pressure in charitable ghing. 0 ) (con 127(01-56. 24. Cain DM. Dane J. Newman GE 0014) Giving versus giving in. Aced84,W Ann (Mk 505-533. 25. Rand DG, Greene JO, Novak MA (2012) Spontaneous giving and calculated greed. Name 4890416)427-m 26. Nielsen UN. Titan JR. Wengstrarn E 0014) Second thoughts on free riding. (con Len 122128136-139. 27. Rubinstein A (2014) A typology of Miners, Between instinc6ve end contemplative. Available at www.arlelrublmtein.tau.acilfpapersrtypology.pdf. Accessed January 15,2015. 28. Rend 0G. Epstein 26 12014) Risking your life without e second thought Intuitive decisionmatung and extreme altrtism. Available at suncomiabstracte2424036. Accessed January IS, 2015. 29. Simon HA (1955) A Wier:aced model of ratio.* choke. 0 ) (con 690119-I It 30. Trotsky A. Kahneman D (1974) Judgment under uncertainty Heuristes and biases. Science 18501571:1124-1131. 31. Kahnemen D. Slavic P. Tvenky A 4191la Aelpment Wider Uncertainty: Heuristic and Biases (Cambridge WIN Press, Cambridge, United Kingebm). 32. Rend D. et el. (2013) Intuitive cooperetkm end the social heseistks hypethesh: Evidence from 15 tine constraint studies. Available et sun comfebstred4222683. Accessed January IS, 2015. 33. Rend EIG, et el. (2014) Social heuristks shone intuitive cooperation. fiat Continuo 51201483677. 34. Tursky B, Shapiro D. Older A, Kahneman D11969) Funnier/ heart rate, and skin re• 'Mane Menges during a mental talk. Et‘o hydro( 79113:164-167. 35. Pinker 5 (1997) How the *NM Weds (NortoA New York). 36. Catcher CR. Mbar Y. Paarro DA (2013) How quick decisions aire,Mbte moral char. ester. Soc hydro! Personal So 401108-315. 37. Feb E, fischbather U 0003) The neon of human attruern. Nana 425169608785-791. 6 of 6 ' www.pnastaggfcgifdo110.1073rptes.1417904112 38 Deity" AW. Kresnow MM. Cosmides 1. Took 1120111 Evolution of direct rmitsrodlY under uncertainty can explain human gemoosity n oneshot encounters. Proc Nail Aced ()SA 10802)13335-13340. 39 Lin It Ong D1201110mersingaltrtionv Type Preferences in the Leboretory. Awrbbre et ices.gmu.odunsp-contentkoploadv2012/01/Deserving-MtruernType.Preferences.in. theleboratorphy.Ongond-Lin pdf. Accessed January 12.2015. 40 Baler Fl. Rodriguez A Ow MG. Mershell L (1995) The relative effechkeness of a deed request message and a megMrig message on friends and strangen. Corn- manic Res 22(4475-484 41. Mills J. Clark MS 11994) Communal and exchange relationships: Controversies and research. Theoretical Frameworks for Personal liotationsNps. ads Mbar R. Gilmour R (Lawrence ErBoeurn Assodeles. Inc. Hillsdele. NO 42. shackellowdllA Buss DM (1996) Betrayal In rnatedups, friends/roc and coalitions. Pen Soc PlyChOrika22(113:1151-1164. 43. Silk /8 (2003) Cooperation without counting- The puzzle of friendship. Genetic and CuMmai froamon of CooparatiorL ed Hammerstein P (1ST Press, carnerkga, MAL pp 37-54. 44 Pinker S. Nowak MA Lee JJ Wag The logic of indirect speech. Prot Nat Aced ki USA 10501833-838. 4S. Fide AP 11992) The four elementary harms of sociality framework for a united theory of nodal rotations. Psycho/Rev 99(4)489-723. 46. Kant I (2002) Groundwork for the Metaphysic, of Mora ed Wood AN (Yale UAW PPM, Hew Haven, CO. 47. Tetlo PE (2003) Thinking the uMhintet4e: Sacred values and taboo cognitions Trends Coen Sc) 717/320-324. 48. fenhtman C. Gaiety U, Hoffman M 120111 Taboos and identity: Considering the unthinkable. An (con I Afivoeten 312)119-164. 49. Frank RH 41911111 Passion Within Peyton: The Strategic Rob of the Emotions PAW Horton 16 Co., Hew York). 50. Schelling TC (19030) The Strategy of Confect eleverd Univ Prep, Cambridge. MN. 51. Hirshleiter I (1967)On the Emotions as Gulf antort Of Threat and ProrMeriMIT PIO% Cambridge, MN 52 VAnten E 12014/ feeling Smart. Why Our &notions Are More Rational Then We Think (Public Athos. New York). 53. Ekman P. Sorenson ER Friesen VN (1969) Pal cultural elements In facial denims of emotion. Science 164(36751'.86-88. SA Hunan Pi Davidson N. Friesen WV119901 The Mechem» smile: Emotional repression and bah physiology. II. / Pars Soc Psycho/ 58(2/342-353. SS. Reed It Zeglen KN. Schmidt Ki. (2012) Facial expressions es honest Pgnels of co. operetta Intent In a one-shot anonymous prisoners dilemma game. (vol Nun, !May 330)200-209. 56 Reed M DeSciali P. Pinker SA (2014) The commitment function of eery facial expressions. Psycho' So 230):1511-1517. Hoffman et aL EFTA00632636