Decision Science News

FLASH MOB #2 TO POPULATE STACKOVERFLOW.COM WITH R QUESTIONS

>From: The R Flashmob Project
>Subject: R Flashmob #2
>
>You are invited to take part in R Flashmob, the project that makes the
>world a better place by posting helpful questions and answers about the
>R statistical language to the programmer’s Q & A site stackoverflow.com
>
>Please forward this to other people you know who might like to join.
>
>FAQ
>
>Q. Why would I want to join an inexplicable R mob?
>
>A. Tons of other people are doing it.
>
>Q. Why else?
>
>A. Stackoverflow was built specifically for handling programming questions.
>It’s a better mousetrap. It offers search (and is well indexed by search engines),
>tagging, voting, the ability to choose the “best” answer to a question, and the ability to
>edit questions and answers as technology progresses. It has a karma system to
>reward people who are happy to help and discourage MLJs (mailing list jerks).
>
>Q. Do the organizers of this MOB have any commercial interest in stackoverflow?
>
A. None at all. We’re just convinced it is the best way to help and promote R. All
>the content submitted to stackoverflow is protected by a Creative Commons
>CC-Wiki License, meaning anyone is free to copy, distribute, transmit, and
>remix the information on stackoverflow. All the content on stackoverflow is
>regularly made available for download by the public.
>
>INSTRUCTIONS – R MOB #2
>Location: stackoverflow.com
>Start Date: Tuesday, September 8th, 2009
>Start Time:
>10:04 AM – US Pacific
>11:04 AM – US Mountain
>12:04 PM – US Central
>1:04 PM – US Eastern
>6:04 PM – UK
>7:04 PM – Continental W. Europe
>5:04 AM (Weds) – New Zealand (birthplace of R)
>Duration: 50 minutes
>
>(1) At some point during the day on September 8th, synchronize your watch to
>http://timeanddate.com/worldclock/personal.html?cities=137,75,64,179,136,37,22
>
>(2) The mob should form at precisely 4 minutes past the hour and not beforehand.
>
>(3) At 4 minutes past the hour, you should arrive at stackoverflow.com, log in,
>and post 3 R questions. Be sure to tag the questions “R”. See the posting
>guidelines at http://stackoverflow.com/faq to understand what makes a good
>question.
>
>(4) Follow R Flashmob updates at http://twitter.com/rstatsmob
>
>(5) Post twitter messages tagged #rstats and #rstatsmob during the mob,
>providing links to your questions.
>
>(6) During the R MOB, you can chat with other participants on the #R channel
>on IRC (freenode). To do this, install the Chatzilla extension on Firefox.
>Click “freenode” on the main screen. Then type /join #R in the field at the
>bottom of the screen. Then chat.
>
>(7) If you finish posting your three questions within the 50 minutes, stick
> around to answer questions and give “up votes” to good questions and answers.
>
>(8) IMPORTANT: After posting, sign the R Flashmob guestbook at
>http://bit.ly/6F8B2
>
>(9) Return to what you would otherwise have been doing. Await
>instructions for R MOB #3.

(Don’t know R yet? Learn by watching: R Video Tutorial 1, R Video Tutorial 2)

PROBABILISTIC INFORMATION ON WHETHER YOUR FLIGHT WILL BE LATE

DSN reader Yael sends along this NY Times InTransit piece on Flight Caster, a web site that uses historical data to generate probabilities that flights will be late. DSN likes the following things:

• The idea of FlightCaster
• The idea of ubiquitous probabilistic information in the age of the networked database and smartphone
• The idea that accurate probabilistic information in the environment will encourage the public to embrace and understand  probabilities

We thought Flight Caster would be useful this week, as Decision Science News is travelling to the SPUDM: Subjective Probability, Utility, and Decision Making Conference in Italy, but unfortunately Flight Caster does not do flights outside the US.

However, it does remind the DSN Editor of the time in the 1990s when, just before a trip to Milan, he asked an Italian colleague what percentage of Alitalia flights were late. “What percentage?” The colleague blinked and leaned forward to make sure he didn’t misunderstand. “Hundred!” he laughed, looking like he’d just been asked to add ninety-nine and one.

THE DECISION OF HOW TO SPEND TIME

The neatest visualization of the last 30 days has to be this New York Times interactive graphic showing how different groups of people decide to spend their time. It is based on data from the American Time Use Survey.

Play with it and read the related article. Decision Science News assures you it will be worth your time. Ha.

Wouldn’t it be great if the talented folks who design and program these things would team up with decision scientists to design interactive experiments that are their own reward? See our attempts here. Not as impressive as the above, but a start.

THE OXFORD HANDBOOK OF ANALYTICAL SOCIOLOGY

With millions of people connecting in online social networks, researchers across academia are racing to make sense of the terabytes of new social data being generated daily.

Some folks, like Duncan Watts, have been thinking about social networks for a long time, even before the online variants went viral. In fact, throughout the 20th century, the field of analytical sociology has been developing tools to prepare us for where we stand now.Those analytical sociologists: They’ve skated to where the data puck is going and brought us closer to the goal of understanding the social net. (Four hockey allusions in one sentence is a DSN record – Ed.)

You can learn all about it in the The Oxford Handbook of Analytical Sociology
(publisher’s info) and get started by reading the free online chapter What is analytical Sociology all about? An introductory essay by Peter Hedström and Peter Bearman.

TABLE OF CONTENTS

Foundations
1: Peter Hedström and Peter Bearman: What is analytical sociology all about? An introductory essay by Peter Hedström
2: Peter Hedström and Lars Udéhn: Analytical sociology and theories of the middle range
Social Cogs and Wheels
3: Jon Elster: Emotions
4: Jens Rydgren: Beliefs
5: Jeremy Freese: Preferences
6: Trond Petersen: Opportunities
7: Dan Goldstein: Heuristics
8: Diego Gambetta: Signaling
9: Jon Elster: Norms
10: Karen Cook and Alexandra Gerbasi: Trust
Social Dynamics
11: Michael Macy and Andreas Flache: Social dynamics from the bottom up: Agent-based models of social interaction
12: Elizabeth Bruch and Robert Mare: Segregation dynamics
13: Michael Biggs: Self fulfilling processes
14: Matthew Salganik and Duncan Watts: Social influence: The puzzling nature of success in cultural markets
15: Yvonne Åberg: The Contagiousness of Divorce
16: Katherine Stovel and Christine Fountain: Matching
17: Delia Baldassarri: Collective action
18: Meredith Rolfe: Conditional choice
19: James Moody: Network dynamics
20: Duncan Watts and Peter Dodds: Threshold models of social influence
21: Christopher Winship: Time and scheduling
22: Scott Feld and Bernard Grofman: Homophily and the focused organization of ties
23: Joel Podolny and Freda Lynn: Status
24: Ivan Chase and W. Brent Lindquist: Dominance hierarchies
25: Stathis Kalyvas: Conflict
Perspectives from other fields and approaches
26: Richard Breen: Game theory
27: Iris Bohnet: Experiments
28: Hannah Brueckner: Surveys
29: Diane Vaughan: Analytical ethnography
30: Karen Barkey: Historical sociology

DRIVEN TO DISTRACTION

The headline E-mails ‘hurt IQ more than pot’ caught our attention here at DSN. Reading the article, we see that the study is not about intelligence as a trait being affected by internet interruptions. It simply uses an IQ test as a measure, we suppose, of being able to think clearly.

In any case, it is a worthwhile topic, and one that will only get more important as time goes on. Do the distractions of working in a networked world prevent us from reasoning well and making good decisions? Stanford prof Jeffrey Pfeffer argues this in his comment Stop Working for Technology – Make It Work For You.

This ties back into our favorite topics of defaults and information design. Most people don’t change the default settings when they install software. If one person has a default browser homepage that puts out constant interruptions (e.g., news flashes, email inbox, portfolio updates, etc), and another person has one that promotes getting work done (e.g., a featureless search engine box), who will get more work done? Who will feel better about what they have accomplished at the end of the week? Decision Science News is working on this topic and has some studies in the can.

USING GRAPHICAL DISPLAYS TO COMMUNICATE TERRORISM RISKS

Jon Baron sent over this timely article http://news.bbc.co.uk/1/hi/magazine/8153539.stm

I like the way the author suggests visual representations (as above) to make the point. That particular representation seems vaguely familiar. I wonder if the author knows there’s been quite a bit of research done on it.

1) Sedlmeier, P., & Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General, 130, 380–400.

2) Galesic, M., Garcia-Retamero, R., & Gigerenzer, G. (2009). Using icon arrays to communicate medical risks to low-numeracy people. Health Psychology, 28(2), 210-216.

3) This one just in via the comments:
Hawley, S.T., Zikmund-Fisher, B., Ubel, P., Jancovic, A., Lucas, T., Fagerlin, A. (2008). The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Education and Counseling, 73, 448-55.

INCENTIVES TO STATE PROBABILITIES OF BELIEF TRUTHFULLY

We have all been there. You are running an experiment in which you would like participants to tell you what they believe. In particular, you’d like them to tell you what they believe to be the probability that an event will occur.

Normally, you would ask them. But come on, this is 2009. Are you going to leave yourself exposed to the slings and arrows of experimental economists? You need to give your participants an incentive to tell you what they really believe, right?

Enter the scoring rule. You pay off the subjects based on the accuracy of the probabilities they state. You do this by observing some outcome (let’s say “rain”) and you pay a lot of money to the people who assigned a high probability to it raining and you pay a little money (or even impose a fine upon) those who assigned a low probability to it raining. A so-called “proper” scoring rule is one in which people will do the best for themselves if they state what they truly believe to be the case.

Three popular proper scoring rules are the Spherical, Quadratic, and Logarithmic. Let’s see how they work.

Suppose in your experimental task you give people the title of a movie, and they have to guess what year the movie was released.  You tell them at the outset that the movie was released between 1980 and 1999: that’s 20 years. So you have these 20 categories (years) and you want people to assign a probability to each year. Afterwards, you will pay them out based on the actual year the movie was released and the probability they assigned to that year.

Let r be the vector of 20 probabilities, and r_1 could be the probability they assign to 1980 being the year of release, and r_2 the probability that it was 1981, so on through r_20 for 1999’s probability. Naturally, all the r’s add up to one, as probabilities like to do. Now, let r_i be the probability they assign to the year which turns out to be correct.

Under the Spherical scoring rule, their payout would be r_i / (r*r)^.5

Under the Quadratic scoring rule, the payout would be 2*r_i – r*r

Under the Logarithmic scoring rule, the payout would be ln(r_i)

In the movie above, the top row shows various sets of probabilities someone might assign to the 20 years. (Imagine the categories along the x-axis are the years 1980 to 1999).  Each bar in the graphs in the bottom three rows shows the person’s payout if that year turns out to be correct, based on the probabilities assigned to each year in the top row.

As you can see, when they assign a high probability to a category and it turns out to be correct, their payout is high. When they assign a low payout to a category and it turns out to be correct, their payout is low.

You’ll notice that the Logarithmic scoring rule goes right off the bottom of the page. This is because the log of small probabilities are negative numbers far beneath zero, and the log of 0 is negative infinity!

While I was at Stanford I heard that decision scientist extraordinaire Ron Howard (no relation) used to make students assign probabilities to the alternatives (A, B, C or D) on the multiple choice items on the final exam. The score for each question was the log of the probability they assigned to the correct answer. This means, of course, that if you assign a probability of 0 to alternative “B” and alternative “B” turns out to be correct, your score on that question is negative infinity. I always wondered if you got a negative infinity on one question if it meant you got negative infinity on the exam, or if there was some mercy clause.

But the main reason I am writing this post is because I wonder what experimental economists and psychologists are supposed to do when implementing log scoring rules in the lab. Naturally, you can endow the participant with cash at the beginning of the experiment and have them draw down with each question, but what do you do if they score a negative infinity? Take their life savings?

Winkler (1971) decided that he would treat probabilities less than .001 as .001 when it came time to imposing the penalty. Does anyone know of other methods?

REFERENCE

Robert L. Winkler (1971)  Probabilistic Prediction: Some Experimental Results, Journal of the American Statistical Association, Vol. 66, No. 336.  pp. 675-685.

NOTE

To make this simulation, I’ve drawn on the top row various beta distributions of differing modes between two fixed endpoints. This is akin to having a min and a max guess for the year of release, then entertaining various years between those two endpoints as most likely.

ARE WE IN CONTROL OF OUR OWN DECISIONS?

Society for Judgment and Decision Making president Dan Ariely gave a TED talk on decision making, which they recently posted on their site. The decision making society president gives a talk on decision making: What could be more relevant for Decision Science News, which, after all is a website about decision research in Marketing, Psychology, Economics, Medicine, Law, Management, Public Policy, Statistics, Computer Science & Interaction Design.

Note the bit on organ donation at the five minute mark, it’s another favorite topic here at DSN.

If you are not familiar with the TED site, it’s a great source of mind-expanding lectures. Attending the TED conference costs a small fortune, but fortunately the content on their website is free. (You can even download copies to your own computer.)

MAKING LUCK WORK FOR YOU

Decision Science News was just at the ESMT Annual Forum in Berlin where we spoke in a session with Martin Weber, Gerd Gigerenzer, Stephan Meier, Luc Wathieu and Robin Hogarth and suddenly remembered that Hogarth, along with INSEAD’s Spyros Makridakis and Anil Gaba, has a new book out called Dance with Chance: Making Luck Work for You.

The book’s Web site has a number of excerpts for free download:

• Preface (pdf)
• The illusion of control from Chapter 1: Three Wishes from a Genie
• Some national puzzles from Chapter 2: The Ills of Pills
• Mind over medicine from Chapter 3: Getting the Right Medicine
• The power of luck from Chapter 5: Watering Your Money Plant
• Mediocrity and failure from Chapter 6: Lessons from Gurus
• The creative destruction of copper from Chapter 7: Creative Destruction
• Take a chance on me from Chapter 8: Does God Play Dice?
• The statistician who ate humble pie from Chapter 9: Past or Future
• A black Monday and a black swan from Chapter 10: Of Subways and Coconuts – Two Types of Uncertainty
• Blinking marvelous from Chapter 11: Genius or Fallible?
• Predicting marital happiness from Chapter 12: The Inevitability of Decisions
• Increasing the sum of human happiness from Chapter 13: Happiness, Happiness, Happiness

Also fun is this video of the authors chatting with Nassim Taleb. Anil Gaba makes a point in the video which, coincidentally, was Decision Science News’ thesis in our ESMT Annual Forum talk on navigating turbulent times.

“I think one thing that is very, very key is to remove this unwarranted respect for sophisticated models where you get taken in by the technical wizardry and you forget the assumptions”

AND WHY ARE THERE MORE MEN THAN WOMEN IN WESTERN US CITIES?

Since seeing this map, Decision Science News can’t quite figure it out. Why do the surpluses of men and women look as they do? What’s up with California?

Source of map:

http://creativeclass.com/whos_your_city/maps/#The_Singles_Map