The "Prisoner's Dilemma" refers to a game theory problem developed in the 1950's. Two prisoners are taken and interrogated separately. If either of them confesses and betrays the other person - "defecting" - they'll receive a reduced sentence, and their partner will get a greater sentence. However, if both defect, then they'll both receive higher sentences than if neither of them confessed.
This brings the prisoner to a strange problem. The best solution individually is to defect. But if both take the individually best solution, then they'll be worst off overall. This has wide ranging implications for international relations, negotiation, politics, and many other fields.
Members of LessWrong are incredibly smart people who tend to like game theory, and debate and explore and try to understand problems like this. But, does knowing game theory actually make you more effective in real life?
I think the answer is yes, with a caveat - you need the basic social skills to implement your game theory solution. The worst-case scenario in an interrogation would be to "defect by accident" - meaning that you'd just blurt out something stupidly because you didn't think it through before speaking. This might result in you and your partner both receiving higher sentences... a very bad situation. Game theory doesn't take over until basic skill conditions are met, so that you could actually execute any plan you come up with.
The Purpose of This Post: I think many smart people "defect" by accident. I don't mean in serious situations like a police investigation. I mean in casual, everyday situations, where they tweak and upset people around them by accident, due to a lack of reflection of desired outcomes.
Rationalists should win. Defecting by accident frequently results in losing. Let's examine this phenomenon, and ideally work to improve it.
Contents Of This Post
I know a lot of people who were nerds in school, and they all tell the same story: there is a strong correlation between being smart and being a nerd, and an even stronger inverse correlation between being a nerd and being popular. Being smart seems to make you unpopular.[...]The key to this mystery is to rephrase the question slightly. Why don't smart kids make themselves popular? If they're so smart, why don't they figure out how popularity works and beat the system, just as they do for standardized tests?[...]So if intelligence in itself is not a factor in popularity, why are smart kids so consistently unpopular? The answer, I think, is that they don't really want to be popular.If someone had told me that at the time, I would have laughed at him. Being unpopular in school makes kids miserable, some of them so miserable that they commit suicide. Telling me that I didn't want to be popular would have seemed like telling someone dying of thirst in a desert that he didn't want a glass of water. Of course I wanted to be popular.But in fact I didn't, not enough. There was something else I wanted more: to be smart.
From when I was still forced to attend, I remember our synagogue's annual fundraising appeal. It was a simple enough format, if I recall correctly. The rabbi and the treasurer talked about the shul's expenses and how vital this annual fundraise was, and then the synagogue's members called out their pledges from their seats.
Let me tell you about a different annual fundraising appeal. One that I ran, in fact; during the early years of a nonprofit organization that may not be named. One difference was that the appeal was conducted over the Internet. And another difference was that the audience was largely drawn from the atheist/libertarian/technophile/sf-fan/early-adopter/programmer/etc crowd. (To point in the rough direction of an empirical cluster in personspace. If you understood the phrase "empirical cluster in personspace" then you know who I'm talking about.)
I crafted the fundraising appeal with care. By my nature I'm too proud to ask other people for help; but I've gotten over around 60% of that reluctance over the years. The nonprofit needed money and was growing too slowly, so I put some force and poetry into that year's annual appeal. I sent it out to several mailing lists that covered most of our potential support base.
And almost immediately, people started posting to the mailing lists about why they weren't going to donate. Some of them raised basic questions about the nonprofit's philosophy and mission. Others talked about their brilliant ideas for all the other sources that the nonprofit could get funding from, instead of them. (They didn't volunteer to contact any of those sources themselves, they just had ideas for how we could do it.)
Now you might say, "Well, maybe your mission and philosophy did have basic problems - you wouldn't want tocensor that discussion, would you?"
Hold on to that thought.
Because people were donating. We started getting donations right away, via Paypal. We even got congratulatory notes saying how the appeal had finally gotten them to start moving. A donation of $111.11 was accompanied by a message saying, "I decided to give **** a little bit more. One more hundred, one more ten, one more single, one more dime, and one more penny. All may not be for one, but this one is trying to be for all."
But none of those donors posted their agreement to the mailing list. Not one.
So far as any of those donors knew, they were alone. And when they tuned in the next day, they discovered not thanks, but arguments for why they shouldn't have donated. The criticisms, the justifications for not donating - only those were displayed proudly in the open.
As though the treasurer had finished his annual appeal, and everyone not making a pledge had proudly stood up to call out justifications for refusing; while those making pledges whispered them quietly, so that no one could hear.
Indeed, that's a problem. Eliezer continues:
"It is dangerous to be half a rationalist."
Our culture puts all the emphasis on heroic disagreement and heroic defiance, and none on heroic agreement or heroic group consensus. We signal our superior intelligence and our membership in the nonconformist community by inventing clever objections to others' arguments. Perhaps that is why the atheist/libertarian/technophile/sf-fan/Silicon-Valley/programmer/early-adopter crowd stays marginalized, losing battles with less nonconformist factions in larger society. No, we're not losing because we're so superior, we're losing because our exclusively individualist traditions sabotage our ability to cooperate.
On Being Pedantic, Sarcastic, Disagreeable, Non-Complimentary, and Otherwise Defecting by Accident
You might not realize it, but in almost all of human civilization it's considered insulting to just point out something wrong someone is doing without any preface, softening, or making it clear why you're doing it.
It's taken for granted in some blunt, "say it like it is" communities, but it's usually taken as a personal attack and a sign of animosity in, oh, 90%+ of the rest of civilization.
In these so-called "normal people's societies," correcting them in front of their peers will be perceived as trying to lower them and make them look stupid. Thus, they'll likely want to retaliate against you, or at least not cooperate with you.
Now, there's a time and place to do this anyways. Sometimes there's an emergency, and you don't have time to take care of people's feelings, and just need to get something done. But surfing the internet is not that time.
I'm going to take some example replies from a recent post I made to illustrate this. There's always a risk in doing this of not being objective, but I think it's worth it because (1) I tend to read every reply to me and carefully reflect on it for a moment, (2) I understand exactly my first reactions to these comments, and (3) I won't have to rehash criticisms of another person. Take a grain of salt with you since I'm looking at replies to myself originally, but I think I can give you some good examples.
The first thing I want to do is take a second to mention that almost everyone in the entire world gets emotionally invested in things they create, and are also a little insecure about their creations. It's extraordinarily rare that people don't care what others' think of their writing, science, or art.
Criticism has good and bad points. Great critics are rare, but they actually make works of creation even in critique. A great critic can give background, context, and highlight a number of relevant mainstream and obscure works through history that the piece they're critiquing reminds them of.
Good critique is an art of creation in and of itself. But bad critique - just blind "that's wrong" without explaining why - tends to be construed as a hostile action and not accomplish much, other than signalling that "heroic disagreement" that Eliezer talks about.
I recently wrote a post titled, "Nahh, that wouldn't work". I thought about it for around a week, then it took me about two hours to think it through, draw up key examples on paper, choose the most suitable, edit, and post it. It was generally well-received here on LW and on my blog.
I'll show you three comments on there, and how I believe they could be subtly tweaked.
> I wizened up,
I don't think that's the word you want to use, unless you're talking about how you finally lost those 20 pounds by not drinking anymore.
FWIW, I think posts like this are more valuable the more they include real-world examples; it's kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations. It would be like reading a history of quantum mechanics or relativity with all mentions of things like the laser or double-slit experiment or Edding or Michelson-Morley removed.
An interesting start, but I would rather see this in Discussion -- it's not fully adapted yet, I think...
Now, I spend a lot of time around analytical people, so I take no offense at this. But I believe these are good examples of what I'd call "accidental defection" - this is the kind of thing that produces a negative reaction in the person you're talking to, perhaps without you even noticing.
#1 is kind of clever pointing out a spelling error. But you have to realize, in normal society that's going to upset and make hostile the person you're addressing. Whether you mean to or not, it comes across as, "I'm demonstrating that I'm more clever than you."
There's a few ways it could be done differently. For instance, an email that says, "Hey Sebastian, I wanted to give you a heads up. I saw your recent post, but you spelled "wisen" as "wizen" - easy spelling error to make, since they're uncommonly used words, but I thought you should know. "Wizen" means for things to dry up and lose water. Cheers and best wishes."
That would point out the error (if that's the main goal), and also engender a feeling of gratitude in whoever received it (me, in this case). Then I would have written back, "Hey, thanks... I don't worry about spelling too much, but yeah that one's embarrassing, I'll fix it. Much appreciated. Anyways, what are you working on? How can I help?"
I know that's how I'd have written back, because that's how I generally write back to someone who tries to help me out. Mutual goodwill, it's a virtuous cycle.
Just pointing out someone is wrong in a clever way usually engenders bad will and makes them dislike you. The thing is, I know that's not the intention of anyone here - hence, "defecting by accident." Analytical people often don't even realize they're showing someone up when they do it.
I'm not particularly bothered. I get the intent behind it. But normal people are going to be ultra-hostile if you do it to them. There's other ways, if you feel the need to point it out publicly. You could "soften" it by praising first - "Hey, some interesting points in this one... I've thought about a similar bias of not considering outcomes if I don't like what it'd mean by the world. By the way, you probably didn't mean wizen there..." - or even just saying, "I think you meant 'wisen' instead of 'wizen'" - with links to the dictionary, maybe. Any of those would go over better with the original author/presenter whom you're pointing out the error to.
Let's look at point #2. "FWIW, I think posts like this are more valuable the more they include real-world examples; it's kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations."
This is something which makes people trying to help or create shake their head. See, it's potentially a good point. But after someone takes some time to create something and give it away for free, then hearing, "Your work would be more valuable if you did (xyz) instead. Your way is kind of odd."
People generally don't like that.
Again, it's trivially easy to write that differently. Something like, "Thanks for the post. I was wondering, you mentioned (claim X), but I wonder if you have any examples of claim X so I can understand it better?"
That one has - gratitude, no unnecessary criticism, explains your motivation. All of which are good social skill points, especially the last one as written about in Cialdini's "Influence" - give a reason why.
#3 - "An interesting start, but I would rather see this in Discussion -- it's not fully adapted yet, I think..."
The difference between complaining and constructive work is looking for solutions. So, "There's some good stuff in here, but I think we could adapt it more. One thing I was thinking is (main point)."
Becoming More Self-Aware and Strategic; Some Practical Social Guidelines
From Anna Salamon's "Humans Are Not Automatically Strategic" -
But there are clearly also heuristics that would be useful to goal-achievement (or that would be part of what it means to “have goals” at all) that we do not automatically carry out. We do not automatically:
- (a) Ask ourselves what we’re trying to achieve;
- (b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;
- (c) Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;
I found you this morning via some sequence of forgotten links that left me at your Lifehacker article.
Excellent post. I found myself highlighting points (using the new Scrible tool) and annotating with my own experiences of things both I have done in the past and have had done to me! Even the most "I tell it like it is" person will have times they don't appreciate receiving "the truth," hmm?
A few things I've found useful as I've softened my edges over the years:
1) writing and rewriting, even the simplest emails/posts/comments, to make sure I'm really saying what I want to say with directness, without the seemingly innocent barbs, and with the appropriate context
2) smilies -- cheesy, perhaps, but helpful in written text when I'm not sure my tone is coming across
3) this book, which is aimed at ADD people, covers some of the same territory. You've reminded me I'm probably due to re-read it.
I love the post--it's really a fantastic how-to guide for getting along in the workplace and building a good reputation. On the other hand, I really hate watering down a solid, technical email with language like "Now, this is only my opinion, but ..." or "I think *some solid facts* but I'm not sure, what do you think?"
In my opinion, most of the techniques which require adding dilutative language are aimed at solely at emotionally immature adults who cannot properly deal with ideas and criticism. In an ideal world, we would work with fully-matured peers.
Unfortunately, it's not an ideal world, and employers are less selective in soft-skills than hard skills.
REALLY liked this one. I mean it, really. One of your best posts. Although I am currently highlighting a potential pitfall... you don't want to be too lavish with your praise, because there is a danger of raising the high water mark, and making yourself come off as insincere, or devaluing your compliment. My personal approach to this is to try to only make true compliments, which means sometimes having to look hard for something good when I want to provide balanced criticism. But, you know, that's not such a bad thing. Then, if you're normally complimenting specific things, when something really gels, you can just say, "Wow, this whole thing is awesome," and it means a lot.
Oh, by the way, you used the word analogy above, where I think you meant acronym. But it's easy to understand such a mistake, with all the time you spend responding to us chatty commenters. Cheers!
Thank you for this post.
I have made all of these errors on multiple occasions without any awareness and with awareness but not understanding. I still make them.
Thanks for providing me with an opportunity to understand.
Great post. I am always glad to read about things regarding the archetype intellectual in society, and this post really defined something I have only recently begun to understand.
That being said, is there an archived link to "Humans Are Not Automatically Strategic" ?
It appears broken, but I'd love to read further.
This rings true for me. Not infrequently, I'll see myself falling into this trap as I say something and try to slip something friendly in first. If I'm commenting online, I have more time to think about what I'm saying and the prefacing feels a lot faker. At this point, I frequently axe the comment altogether.
I have no idea how obvious it is to other people that the complimentary fluff was shoe-horned in, but it *feels* very obvious indeed, and this rubs me completely the wrong way. Likely I just need more practice smoothing the transition.
Just FYI, the "related to" links didn't survive the transition from LessWrong to here.
I really liked this post, and I thought you made quite a few good points here.
...in almost all of human civilization it’s considered insulting to just point out something wrong someone is doing without any preface, softening, or making it clear why you’re doing it.
I found this to be an excellent point, and wanted to share something that a friend passed on to me.
I had criticized my friend about something we were working on. He said thanks, we concluded our conversation, and I thought all was well. Not five minutes later he called back to critique my handling of the situation. He said that while my criticism was valid, the way I phrased it had put him on the defensive. He shared with me the acronym ACE (Appreciation, Correction/Criticism, Encouragement) for creating constructive responses.
Appreciation - To build up the other person for what they've done and contributed; to show them that you care about them and what they do.
Correction - Fairly obvious, to bring out what was incorrect and provide suggestions for improvement.
Encouragement - Take something they have succeeded in, and encourage them to use that as a guide for the future.
I didn't realize that this was a healthy way to go about confronting people. I had always assumed that people saw through the Appreciation*. It turns out that this approach makes the listener open for discussion, because you're not just tearing them down. They see it as constructive criticism instead of blatant negativity.
I am definitely transitioning from the "social skills don't matter" crowd to the "social skills matter a ton" crowd, so I really appreciate the gems in this post Sebastian.
* They do when it's fake, so it has to be genuine.
Another brilliant reply to "Defecting by Accident - A Flaw Common to Analytical People" - this time from a civil servant, who has granted me permission to share this, but asked not to receive credit for obvious reasons :)
Just read your post on Defecting by Accident. I think there's some valuable insight there which fits with my own experience (I'm a junior civil servant in ---).
One tactic I've acquired/developed for the 'someone proposes something unworkable' situation is to: a) agree pleasantly and b) propose something different. People who have practised assuming an air of authority can come across as being more certain/committed to plans than they actually are. I've been pleasantly surprised how happy people are to move on to better ideas; when doing so doesn't involve loss of face. Doesn't work all the time, but it helps. Forgoing the temptation to nitpick requires a bit of willpower but it gets easier, like a diet.
There's a potential Nash equilibrium where you have a better idea, but think 'They seem to believe this strongly, so I'll just accept it and avoid conflict.'
My life philosophy is "Don't be pseudo."
It can be applied everywhere--school, work, relationships, productivity, and health.
It may not be the key to success (this philosophy tends to rub the masses the wrong way), but it's the only way I can live with myself.
I'm not saying I'm any good at my philosophy. People constantly applaud me for being real, brash, and outspoken, but honestly, I'm still constantly pseudo.
I tell myself that I'll do the work today, or that I'm reading a lot, or that I'll start an exercise habit, or that I've been grooming myself daily, but the truth is, I'm not. I'm still a failure.