Decision Science News

US GOVERNMENT GUIDELINES BY AGE, SEX, ACTIVITY LEVEL

Click to enlarge

At Decision Science News, we are always on the lookout for rules of thumb.

Our colleague Justin Rao was thinking it would be useful to express calories as a percentage of daily calories. So instead of a coke being 150 calories, you could think of it as 7.5% of your daily calories. Or whatever. The whatever is key.

This is an example of putting unfamiliar numbers in perspective.

So, we were then interested to see if there would be an easy rule of thumb for people to calculate how many calories per day they should be eating, so that they could re-express foods as a percentage of that.

We found some calorie guidelines on the Web published by the US government. With the help of Jake Hofman, we used Hadley Wickham‘s rvest package to scrape them and his other tools to process and plot them.

The result is above. If you have any ideas on how to fit it elegantly, let us know.

We tried a number of fits. Lines are good for heuristics, so we made a bi-linear fit to the raw data (in points). We’re all grownups reading this blog, so let’s focus on the lines to the right of the peak.

Click to enlarge

Time to make the heuristics. For women, you need about 65 fewer calories per day for every decade after age 20. For men, you need about 105 fewer calories per day for every decade after age 20. Or let’s just say 70 and 100 to keep it simple.

So, if you have an opposite sex life partner (OSLP?), keep in mind that you may need to cut back by more or fewer calories as the person across the table as you age together. Same sex life partner (SSLP?), cut back the same amount. Just don’t go beyond the range of the chart. The guidelines suggest even sedentary men shouldn’t eat fewer than 2,000 calories a day at any age. For women, that number is 1650.

REFERENCES

Barrio, Pablo J., Daniel G. Goldstein, & Jake M. Hofman. (2016). Improving comprehension of numbers in the news. ACM Conference on Human Factors in Computing Systems (CHI ’16). [Download]

The R code, below, has some other attempts at plots in it. You may be most interested in it as a way to see rvest in action. Or just to get the data.

PREDICTION MARKETS NOT AS BAD AS THEY APPEAR

Two recent events in the UK made it look like prediction markets’ predictions aren’t worth much.

The soccer team Leicester City won the premiere league title despite the markets putting the odds of them doing so at 5,000 to 1 (.02%).

Last week, people in the UK, voted to leave the European Union. A few hours before it was sure they would exit, a prediction market put their probability of leaving at 10%. See the figure above from PredictIt. X axis is roughly time before the outcome was certain. Y axis can be interpreted as probability of exit (70 cents = 70%).  It jumped from 10% to 90% in just five hours.

Analysts like to “explain” market results, coming up with a reason why an event was a failure of the prediction market. For instance, in the two events above, the Wall Street Journal, perhaps correctly, claims the bets were unduly influenced by London bettors. Through big London bets the odds moved to reflect what Londoners believe instead of the sentiment of the crowd. In predicting a Brexit, the sentiment of the crowd is exactly what you want.

Whenever the prediction market is far on the wrong side of 50%, explanations will arise as to why the prediction market was wrong. Let’s take a step back here.

A desirable property of a prediction market is that it is calibrated. To be calibrated, events that it predicts to be 90% likely should occur 90% of the time. Events that it predicts to be 10% likely should occur 10% of the time.

If events that it predicts to be 10% likely (e.g. Brexit) occur 0% of the time, the prediction market has a problem. It is over-estimating the chances.

Looking at the calibration of prediction markets across many events, one will see that they are typically very well calibrated. Take, for instance that Hypermind prediction market. The figure below shows close to 500 events that it predicted. If the market were perfectly calibrated, the points would fall along the diagonal line (i.e., 10% likely events would happen 10% of the time, 20% likely events would happen 20% of the time, and so on).

It’s very well calibrated. Here’s a similar chart for U.S. primaries at PredictWise.

So the next time a market “misses” a 10% prediction, let us keep in mind that it needs to miss 10% of those predictions to stay calibrated. As the Émile Servan-Schreiber mentions in this post, “It is perhaps a bit ironic to note that the data from the Brexit question slightly improved [the prediction market’s] overall calibration. It is as if the occurence of an unlikely event was long overdue in order to better match predicted probabilities to observed outcomes!”

You may wish that that prediction markets had prefect calibration and predicted only 0% or 100%. We do, too. But the world is an imperfect place.

WE THOUGHT IT WAS NUMBER ONE

Robert Meunchen has done a rather in depth analysis of the popularity of various packages for statistical computation. More detail here.

We were surprised to see that R has passed SAS for scholarly use. We were surprised because we assumed this would have happened years ago. Meunchen too predicted it would have happened in 2014.

In addition, we were surprised to see that SPSS holds the number one spot, and by a fair margin. We can only think of one person who uses SPSS.

Perhaps we’re guilty of false consensus reasoning.

A chart further down the post shows that Python is rapidly growing in use. That meshes with what we observe. Many of our PhD student interns come in using Python for data analysis. One who I worked with, Chris Riederer, is even writing a version of dplyr in Python and calling it dplython, of course.

TESTING A HEURISTIC TO ESTIMATE STANDARD DEVIATION

Click to enlarge

Say you’ve got 30 numbers and a strong urge to estimate their standard deviation. But you’ve left your computer at home. Unless you’re really good at mentally squaring and summing, it’s pretty hard to compute a standard deviation in your head. But there’s a heuristic you can use:

Subtract the smallest number from the largest number and divide by four

Let’s call it the “range over four” heuristic. You could, and probably should, be skeptical. You could want to see how accurate the heuristic is. And you could want to see how the heuristic’s accuracy depends on the distribution of numbers you are dealing with.

Fine.

We generated  random numbers from four distributions, pictured above. We nickname them (from top to bottom): floor, left of center, normalish, and uniform. They’re all beta distributions. If you want more detail, they’re the same beta distributions studied in Goldstein and Rothschild (2014). See the code below for parameters.

We vary two things in our simulation:
1) The number of observations on which we’re estimating the standard deviation.
2) The distributions from which the observations are drawn

With each sample, we compute the actual standard deviation and compare it to the heuristic’s estimate of the standard deviation. We do this many times and take the average. Because we like the way mape sounds, we used mean absolute percent error (MAPE) as our error metric. Enough messing around. Let’s show the result.

Click to enlarge

There you have it. With about 30 to 40 observations, we could get an average absolute error of less than 10 percent for three of our distributions, even the skewed ones. With more observations, the error grew for those distributions.

With the uniform distribution, error was over 15 percent in the 30-40 observation range. We’re fine with that. We don’t tend to measure too many things that are uniformly distributed.

Another thing that set the uniform distribution apart is that its error continued to go down as more observations were added. Why is this? The standard deviation of a uniform distribution between 0 and 1 is 1/sqrt(12) or 0.289. The heuristic, if it were lucky enough to draw 1 and a 0 as its sample range, would estimate the standard deviation as 1/4 or .25. So, the sample size increases, the error for the uniform distribution should drop down to a MAPE of 13.4% and flatten out. The graph shows it is well on its way towards doing so.

REFERENCES

Browne, R. H. (2001). Using the sample range as a basis for calculating sample size in power calculations. The American Statistician, 55(4), 293-298.

Hozo, S., Djulbegovic, B., & Hozo, I. (2005). Estimating the mean and variance from the median, range, and the size of a sample. BMC medical research methodology, 5(1), 1.

Ramıreza, A., & Coxb, C. (2012). Improving on the range rule of thumb. Rose-Hulman Undergraduate Mathematics Journal, 13(2).

Wan, X., Wang, W., Liu, J., & Tong, T. (2014). Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC medical research methodology, 14(1), 135.

Want to play with it yourself? R Code below. Thanks to Hadley Wickham for creating tools like dplyr and ggplot2 which take R to the next level.

OR AT LEAST WITH SHORTER SENTENCES

If you are an academic, and you probably are if you read this blog, you may be interested in tools that improve writing. Recently, we stumbled upon the Hemingway Editor, a tool to encourage simple and forceful writing. It flags hard to read (i.e., long) sentences. It also flags adverbs, passive voice and unnecessarily complex phrases. We think that following all the editor’s advice would be a bad idea. Yet, using it for a few paragraphs on a paper reminds you to keep things simple. And it does, at least superficially, make you write more like Hemingway.

For fun, we applied it to one of our own abstracts below. It went from 145 words with grade 18 readability to 119 words with grade 14 readability.

You should beware, however, that readers may infer that simpler academic texts are of lower quality (Galak and Nelson, 2011).

REFERENCES

Galak, J. & Nelson, L. D. (2011). The virtues of opaque prose: How lay beliefs about fluency influence perceptions of quality. Journal of Experimental Social Psychology, Volume 47, Issue 1, January 2011, Pages 250–253

Goldstein, Daniel G., & David Rothschild. (2014). Lay understanding of probability distributions. Judgment and Decision Making, 9(1), 1-14.

Hemingway Editor: http://www.hemingwayapp.com/

BEFORE (145 Words. Grade 18 readability)

How accurate are laypeople’s intuitions about probability distributions of events? The economic and psychological literatures provide opposing answers. A classical economic view assumes that ordinary decision makers consult perfect expectations, while recent psychological research has emphasized biases in perceptions. In this work, we test laypeople’s intuitions about probability distributions. To establish a ground truth against which accuracy can be assesed, we control the information seen by each subject to establish unambiguous normative answers. We find that laypeople’s statistical intuitions can be highly accurate, and depend strongly upon the elicitation method used. In particular, we find that eliciting an entire distribution from a respondent using a graphical interface, and then computing simple statistics (such as means, fractiles, and confidence intervals) on this distribution, leads to greater accuracy, on both the individual and aggregate level, than the standard method of asking about the same statistics directly.

AFTER (119 Words. Grade 14 readability)

How accurate are laypeople’s intuitions about probability distributions of events? The economic and psychological literatures provide opposing answers. A classical economic view assumes that ordinary decision makers consult perfect expectations. Recent psychological research has emphasized biases in perceptions. In this work, we test laypeople’s intuitions about probability distributions. We control the information seen by each subject to establish unambiguous ground truth answers. We find that laypeople’s statistical intuitions are accurate but depend upon the elicitation method. We computed simple statistics from distributions elicited using a graphical interface. The statistics included means, fractiles, and confidence intervals. All statistics derived from distributions were more accurate than those obtained by direct asking. This was true on both the individual and group level.

SOCIETY FOR JUDGMENT AND DECISION MAKING NEWSLETTER

The quarterly Society for Judgment and Decision Making newsletter can be downloaded from the SJDM site:

http://sjdm.org/newsletters/

best,
Dan Goldstein
SJDM President & Newsletter Editor

EVERYTHING YOU EVER WANTED TO KNOW ABOUT THE AMA INTERVIEWS (2016 edition)

The old (above) and new (below) AMA logos

PhD students in Marketing, Psychology, and Economics should send their “packets” out by the fourth of July in the hopes of lining up interviews at the annual AMA Summer Conference. Each year DSN reprints this sort of “what to expect while you’re applying” guide, first published here by Dan Goldstein in 2005. In past years, Dave Hardisty and Abby Sussman have co-authored this guide with Dan to bring it up to date. “I” will refer to Dan in what follows.

SHARE YOUR OWN AMA TIPS
I am more than happy to publish AMA tips, updated information, or just AMA horror stories as part of this post. You can reach me at dan at dangoldstein dot com and let me know if you want to be anonymous or nonymous.

WHY AM I WRITING THIS?
I have seen the Marketing job market turn happy grad students into quivering masses of fear. I want to share experiences and provide a bit of advice to make the whole process less mysterious.

WHY SHOULD ANYONE LISTEN TO ME?
Dan went on the AMA job market in the mid 2000s. Dave and Abby, who thoroughly revised this guide, went on the market in 2012. As a professor, Dan’s conducted scores of AMA interviews and made dozens of hiring decisions. Together, we’ve been on the candidate end many AMA interviews, and experienced numerous campus visits, face-to-face interviews, offers, and rejections.

HOW TO GET INTO THE AMA JOB MARKET
First, at least a couple months before the conference, find where it will be. It’s called the American Marketing Association Summer Conference. Insiders just call it “The AMA”. Get yourself a room in the conference hotel, preferably on the floor where the express elevator meets the local elevator for the upper floors. You’ll be hanging out on this floor waiting to change elevators anyway, so you might as well start there. Next, create a list of schools at which you would like a job. You can find the top 100 schools ranked by journal publications at: http://jindal.utdallas.edu/the-utd-top-100-business-school-research-rankings . Next, apply to each one you’d go to. In the past, this involved physically mailing application packets, but these days nearly everything is electronic. Some schools have online application portals, and others will take applications by email. How do you find out which one? Hopefully, the school you are applying to will have posted a job at ELMAR (see http://ama-academics.communityzero.com/elmar; and subscribe to their mailing list) or elsewhere, which will specify how to submit your application. Other times, you’ll need to get in touch with a department administrator to find out how to apply. It’s a good idea to apply to schools you like even if they say they aren’t hiring. Sometimes things change suddenly (a tenured faculty member unexpectedly moves to another school), and the only thing you risk is your time.

On this topic, one recent market participant wrote us saying: “Sometimes during an AMA interview the people you are interviewing with might tell you up front that they are not planning to hire someone in CB (consumer behavior) and that they are just interviewing you because they wanted to hear about your work (or, more generally, to get to know the star candidates on the market). This happened to me with at least one school, and it almost makes you NOT want to try seriously (i.e., to use that interview as a “break” from the more consequential interviews). However, I would strongly advise candidates to take all their interviews seriously, even with schools that claim not to be hiring CB candidates. First of all, the interview is an opportunity for schools to learn about candidates’ research and therefore an additional opportunity to invest in one’s reputation. But, more importantly, sometimes they end up changing their minds and giving you a fly-out anyway, despite what they initially said about not hiring CB candidates (this happened to me).”

http://www.marketingphdjobs.com/ has general information about the Marketing job market, including a job board showing who is advertising jobs. This site also has information about the times when jobs are announced. Different schools require different materials for the application, involving some combination of: cover letter, CV, teaching statement (and/or teaching evaluations), research statement, letters of recommendation (roughly three), your “job talk” paper, and one or two additional publications.

It’s useful to have your advisor (or if that’s not feasible, his/her assistant) send an e-mail with your materials directly to their colleagues at each of your top choice schools. Ideally you would have your letter writers e-mail their recommendations directly to your advisor so the information is aggregated. This will not take the place of applying through a school’s official online system, but will bring attention to your application before it arrives. This is also useful for schools that are slow to set up their official application, or who are uncertain about whether they’ll be hiring. Some candidates will send hard copies of these packets to all of the schools, although this practice seems to be dwindling.

Note that the CV for AMA takes a special format, including an extended abstract of your “job talk” paper, so you should find an example to model your CV after. It’s helpful if your letters of recommendation come from faculty in marketing. You need to demonstrate your commitment to the field of marketing, so it’s also ideal to have publications in marketing journals, or at least something under review at a marketing journal.

It’s invaluable to have an “application buddy” who is also on the job market. You can share notes about who is hiring, how to apply at each place, etc. It also just makes the whole process more fun, to have a friend along for the ride.

With the advent of electronic submissions, the “marginal cost” of additional submissions is extremely low, and schools are facing a flood of applications. Last year, the median number of applications sent by each candidate was 60, and this will probably increase next year. Each school can only interview a limited number of candidates at AMA (perhaps 20 or 30), so they need to be selective: they are not necessarily looking for the best candidates, but rather the best candidates that would accept an offer from their school (over and above offers from other schools). For this reason, “lower tier” schools will often not give interviews to “upper tier” candidates, because they believe that there is little chance the upper tier candidate would actually accept an offer from them. Therefore, you need to find a way to communicate why you are interested in that school in particular. The best way is if you have a contact at that school (or your advisor has a contact). You should let them know that their school is a high priority for you, and why. Another strategy is to get the word out that you are targeting a particular region (such as the west coast or midwest) or type of school (big city vs rural location, small vs large school, etc). Thus, even though your real priority is probably to get a job *anywhere*, it’s good to specialize a bit to give yourself a competitive advantage. Likewise, once you get an AMA interview or a fly-out, it’s good to have a host of reasons prepared to explain why you are excited about that school in particular. Keep in mind that information about your preferences that you tell to one school may get back to other schools as well. For that reason, it’s best to come up with unique reasons why you like each school that do not detract from your ability to credibly like others.

If you are submitting by email, it’s a good idea to follow up and confirm that they have all your application materials. Sometimes email applications get lost (this happened to a friend of ours at his top-choice school), and once the AMA interviews are scheduled, it may be too late.

As for deadlines: the rough deadline is July 4th, but there is a lot of variability. A few schools have earlier deadlines and will have scheduled all their interviews by the 4th, whereas other schools will be behind schedule and won’t even post a job opening until after the 4th. Within reason (beginning mid-June), there’s an advantage to submitting materials sooner rather than later. Ethan Pew adds “July 4 is still largely the target for sending out packets, however schools seem to be moving to more of a just-in-time process. [In 2011], 53 positions were announced between July 4 and AMA. There were also 24 positions announced the last week in June — and presumably those schools didn’t expect packets by July 4. In total, those 77 positions accounted for 40% of the jobs announced prior to AMA last year.”

THEN WHAT?
Wait to get calls or emails from schools wishing to set up AMA interviews with you. These calls may come in as late as one week before the conference. Often they come when you are sitting outside having a drink with friends. Some schools will not invite you for totally unknown reasons. You may get interviews from the top 10 schools and rejected from the 30th-ranked one. Don’t sweat it. Again, this is the land of total and absolute unpredictability that you’re entering into. Also, know that just because you get an interview doesn’t mean they have a job. Sometimes schools don’t know until the last minute if they’ll have funding for a post. Still, you’ll want to meet with them anyway. Other times, schools are quite certain they have two positions, but then later university politics shift and they turn out to have none.

When the schools call to set up an AMA interview, you will have some flexibility in scheduling. Should you put your top schools at the beginning of the weekend, the end, or somewhere in the middle? Common advice is to put your less preferred schools on the beginning of the first day, because this gives you a chance to practice before the “important” ones. Then you can put your top choice schools early (say 10am, not 8) on the second day of the conference, so you’ve had some time to practice, but you and the interviewers can still be fresh and energized. While it’s nice to schedule strategically, it’s very difficult since you have no idea which schools you’ll ultimately be hearing from, and calls come in over time. Don’t worry if it’s not possible to schedule everything perfectly- there’s no real magic to this. For example, the beginning of the weekend is also when interviewers will be freshest, and a primacy effect could help you: in fact, we know a candidate that put his top school first, and eventually got a job offer from them.

After the AMA, you’ll hopefully get “fly-outs,” that is, offers to come and visit the campus and give a talk. This means you’ve made the top five or so. Offers for fly-outs generally come within a week or two of AMA. Actual job offers start in late October, and the market has generally cleared by Thanksgiving. There’s a second job market that happens after all the schools realize they’ve made offers to the same person. Some schools over-correct for this and don’t make offers to amazing people who would have come. We need some kind of market mechanism to work out this part of the system.

THE “IT’S ALL ABOUT FRIENDSHIP” RULE
Keep in mind that you will leave this process with 1 or 0 jobs. Therefore, when talking to a person, the most likely thing is that he or she will not be your colleague in the future. You should then think of each opportunity as a chance to make a friend. You’ll need friends to collaborate, to get tenure, get grants, and to go on the market again if you’re not happy with what you get. It’s a good idea to send thank-you emails after AMA, to maintain contact, show your interest in the school, and express your appreciation. After all, the AMA interviewers have sacrificed their weekend to talk to you.

HOW DO YOU FIND OUT IN WHICH ROOM TO INTERVIEW?
The schools will send you e-mails either a few days in advance, or the night before telling you which room to go to. Many profs ask the hotel to make their room number public, but for some reason many hotel operators will still not give you the room number.

HOW TO TREAT YOURSELF WHILE THERE
My sponsor gave me the advice of not going out at night and getting room service for breakfast and dinner. This worked for me. Also, the ridiculously high price of a room-service breakfast made me feel like I was sparing no expense, which I found strangely motivating. However, as this guide has gotten more popular, many people are ordering room service breakfast, and there were reports at last year’s AMA that the hotel was overwhelmed with orders and breakfasts were delivered quite late as a result.

HOW DO THE ACTUAL AMA INTERVIEWS GO?
At the prearranged time you will knock on their hotel room door. You will be let into a suite (p=.4) or a normal hotel room (p=.5, but see below). In the latter case, there will be professors with long and illustrious titles—people you once imagined as dignified—sitting on beds in their socks. The other people in the room may not look at you when you walk in because they will be looking for a precious few seconds at your CV. For at least some people in the room, this may be the first time they have concentrated on your CV. Yikes is right. Put the important stuff early in your CV so nobody can miss it. You can expect anywhere between one and nine faculty members to show up usually it will be between three to five. Some of them may take cell phone calls in the middle of your interview. Don’t take It personally.

THE SEAT OF HONOR
There will be one armchair in the room. Someone will motion towards the armchair, smile, and say, “You get the seat of honor!” This will happen at every school, at every interview, for three days. I promise.

THE TIME COURSE
Allow extra time to get to your interview: all the candidates are traveling from one room to another at the same time, and the elevators can get pretty backed up. When you arrive, there will be two minutes of pleasant chit-chat. They will propose that you talk first and they talk next. There will be a little table next to the chair on which you will put your flip book of slides. You will present for 30 minutes, taking their questions as they come. Usually 20 minutes of scripted material will take you through the full time since you’ll be interrupted the whole way through. But you should be prepared to talk for up to 35-40 minutes for quiet groups, or sessions where only one faculty member shows up. They will be very nice. When done, they will ask you if you have anything to ask them. You of course do not. You hate this question. You make something up. Don’t worry, they too have a spiel, and all you need to do is find a way to get them started on it. By the time they are done, it’s time for you to leave. The whole experience will feel like it went rather well.

PREDICTING IF YOU WILL GET A FLY-OUT
It’s impossible to tell from how it seems to have gone whether they will give you a fly-out or not. Again, this is the land of staggering and high-impact uncertainty. They might not invite you because you were too bad (and they don’t want you), or because you were too good (and they think they don’t stand a chance of getting you and they don’t want to waste a precious fly-out on you). The latter fact means that “playing hard to get” is a bad idea. Interviewers will be friendly because everyone wants you to like their school, regardless of whether or not they will invite you for a fly-out.

DO INTERVIEWS DEVIATE FROM THAT MODEL?
Yes.
Sometimes instead of a hotel room, they will have a private meeting room (p=.075). Sometimes they will have a private meeting room with fruit, coffee, and bottled water (p=.025). Sometimes, they will fall asleep while you are speaking (p=.05). Sometimes they will be rude to you (p=.025). Sometimes a key person will miss an early interview due to a hangover (p=.025). Sometimes, if it’s the end of the day, they will offer you alcohol (p=.18, conditional on it being the end of the day).

HOW YOU THINK THE PROCESS WORKS
The committee has read your CV and cover letter and looked at your pubs. They know your topic and can instantly appreciate that what you are doing is important. They know the value of each journal you have published in and each prize you’ve won. They know your advisor and the strengths she or he instills into each student. They ignore what they’re supposed to ignore and assume everything they’re supposed to assume. They’ll attach a very small weight to the interview and fly you out based on your record, which is the right thing to do according to a mountain of research on interviews.

HOW THE PROCESS REALLY WORKS
The interviewers will have looked at your CV for about one minute a couple months ago, and for a few seconds as you walked in the room. They will never have read your entire cover letter, and they will have forgotten most of what they did read. They could care less about your advisor and will get quite annoyed that you didn’t cite their advisor. They’ll pay attention to everything they’re supposed to ignore and assume nothing except what you repeat five times. Flouting 50 years of research in judgment and decision-making, they’ll attach a small weight to your CV and fly you out based on the interview and their gut feeling.

IF ENGLISH IS NOT YOUR MOTHER TONGUE
Your ability to speak English well won’t get you a good job, but your inability to do so will eliminate you from consideration at every top school. Understand that business schools put a premium on teaching. If the interviewers don’t think you can communicate in the classroom, they’re probably not going to take a chance on you. If you are just starting out and your spoken English is shaky, my advice is to work on it as hard as you are working on anything else. Hire a dialect coach (expensive) or an english-speaking actor or improviser (cheaper) to work with you on your English pronunciation. In the Internet age, it’s quite easy to download samples of English conversational speech, for instance from podcasts, for free. It’s also very easy to get a cheap headset and a free audio recorder (like Audacity) with which to practice.

TWO WAYS TO GIVE YOUR SPIEL
1) The plow. You start at the first slide and go through them until the last slide. Stop when interrupted and get back on track.
2) The volley. Keep the slides closed and just talk with the people about your topic. Get them to converse with you, to ask you questions, to ask for clarifications. When you need to show them something, open up the presentation and show them just that slide.
I did the plow the first year and the volley the second year. I got four times more fly-outs the second year. Econometricians are working hard to determine if there was causality. I would not attempt the volley unless you are generally considered to be good with words.
A middle ground is to have a shorter presentation prepared, with many backup slides that you can turn to in response to audience interest or specific questions. This helps the audience understand that you’ve thought about the project deeply, that your responsive to their specific interests and feedback, and that there’s more to support your argument than the prearranged script, but it’s easier for some to execute than a straight volley.
What do you use to show your slides? Many candidates print out slides and put them into a presentation binder, such as one of these: http://tinyurl.com/c46ob64 . Insert two copies of each slide into the binder so you can see a copy from the back, and the audience can see another from the front. Hand-outs are also a good idea, as a supplement to your slides. Bring lots of hand-outs, because it’s common to let them keep one copy. Keep in mind that AMA slides are not the same as a typical PowerPoint presentation. People should be able to read your slides from the back of the room. I recommend no smaller than size 40 font, and larger is better.

Lately, people have talked about showing slides on an iPad, but I don’t know anyone that has actually done this.

HOW TO ACT
Make no mistake, you are an actor auditioning for a part. There will be no energy in the room when you arrive. You have to be like Santa Claus bringing in a large sack of energy. The interviewers will be tired. They’ve been listening to people in a stuffy hotel room from dawn till dusk for days. If you do an average job, you lose: You have to be two standard deviations above the mean to get a fly-out. So audition for the part, and make yourself stand out. If you want to learn how actors audition, read Audition by Michael Shurtleff.

SOCIAL SKILLS MATTER
From the candidate’s point of view, everything is about the CV and the correctness of the mathematical proofs in the job market paper. However, for better or for worse, extra-academic qualities matter. Here are two examples. 1) The Social Lubricant factor. Departments get visitors all the time: guest speakers, visiting professors, job candidates, etc. Some departments are a bunch of folks who stare at their shoes when introduced to a new person. These departments have a real problem: they have nobody on board who can make visitors feel at ease, and sooner or later word starts to spread about how socially awkward the people at University X are. To fix such problems, departments sometimes hire socially-skilled types who know how to make people comfortable in conversation, and who know how to ask good questions during talks. Also, interviewers assume that people who can talk a good game will be star teachers. 2) The Soft Sell factor. Many people succeed in academia not because they are often right, but also because they are masters of making other people feel like they aren’t wrong. Defensiveness or determination to embarrass when responding to critique is an effective way to blow an interview.

HAVE A QUIRK
One of the biggest risks facing you is that you will be forgotten. Make sure the interviewers know something unusual about you. My quirk is that I worked internationally as an actor and theater director for over a decade; I even had a bit part in a Conan O’Brien sketch on TV. It has nothing to do my research, but people always bring up this odd little fact when I do campus visits. Some bits of trivia are just more memorable than others.

DON’T GIVE UP
Never think it’s hopeless. Just because you’re not two SDs above the mean at the school of your dreams, it does not mean you’re not the dream candidate of another perfectly good school.
Many candidates don’t realize the following: The students are competing for schools but the schools are also competing for students. If you strike out, you can just try again next year. I know a person in Psychology who got 70 rejections in one year. I know a person in Marketing who was told he didn’t place in the top 60 candidates at the 20th ranked school. The subsequent year, both people got hired by top 5 departments. One of them is ridiculously famous and considered among the smartest people in Marketing!

RUMORS
Gossip can mess with your chances. Gossip that you are doing well can hurt you because schools will be afraid to invite you if they think you won’t come. Gossip that you are doing poorly can hurt you because schools that like you will be afraid to invite you if they think no one else does. Sometimes people will ask a prof at your school if you would come to their school, and the prof will then ask you. To heck with that. Just say that if they want to talk to you, they should talk with you directly.

The danger of rumors can be summed up by the following story. At ACR I was having a beer with someone who confessed, “you know, my friend X at school Y told me that they want to hire you, but they’re afraid your wife won’t move to Z”. I was single.

MORE PUBLICATIONS ?= LOWER QUALITY

There’s an article out in Science titled The Pressure to Publish Pushes Down Quality. The author’s point is that as the volume of papers goes up, the good ones get lost in the bad ones.

There are other downsides as well. People waste a lot of time funding, writing, reviewing, editing, and reading lots of shoddy papers. The number of papers needed to get tenure keeps going up, which puts a lot of pressure on early-career academics. Lastly, when people have an incentive to publish a lot, they’re not encouraged to thoroughly review what’s come before. If they do, they may learn their idea is not all that new. This can prevent cumulative progress being made in science. As an aside, it’s hard to blame authors for not thoroughly reviewing the literature when so much junk has been published in the last decades.

The annoying thing about the Science article (and this blog post thus far) is that while many agree that increasing pressure to publish is bad, and that there are too many bad papers out there, there aren’t a lot proposed solutions. What should we do about it?

A friend of ours had a suggestion: Only let people publish five papers before tenure.

To be clear, the idea is not to only let people submit five papers for consideration for tenure. The idea is to only let people publish five papers before tenure. If you do, it hurts your case.

Some people hate this idea. But some love it. We can’t help but notice some good sides. It could cause people put more care into the papers they submit. It could reduce the number of papers being submitted for review. It could take the pressure off assistant professors to produce an ever-increasing number of papers before tenure. It could even reduce false alarms, failures to cite past literature, and possibly scientific fraud.

What do you think?

LAST WEEK TONIGHT COVERS BAD SCIENCE

If you have not seen John Oliver’s hilarious rant about bad science from his program Last Week Tonight, you should do so for it is hilarious.

JDMers Joe Simmons, Leif Nelson and Uri Simonshohn should be happy that he uses their term “p-hacking” and even makes a joke about it. What’s the joke? Come on, just watch the video and see.

SOCIETY FOR JUDGMENT AND DECISION MAKING (SJDM) 2016

ANNUAL MEETING OF THE SOCIETY FOR JUDGMENT AND DECISION MAKING, NOVEMBER 18-21, 2016

The Society for Judgment and Decision Making (SJDM) invites abstracts for oral presentations and posters on any interesting topic related to judgment and decision making. Completed manuscripts are not required.

LOCATION, DATES, AND PROGRAM

SJDM’s annual conference will be held in Boston, Massachusetts, November 18-21, 2016. The conference will take place at the Sheraton Boston. Plenary events will include a keynote talk on Sunday, November 20th delivered by Linda Babcock.

SUBMISSIONS: DEADLINE JUNE 20, 2016

The deadline for submissions is June 20, 2016, end of the day. Submissions for symposia, oral presentations, and posters should be made through the SJDM website at http://www.sjdm.org/abstract-review/htdocs Technical questions can be addressed to the webmaster, Jon Baron, at webmaster@sjdm.org. All other questions can be addressed to the program chair, Nina Mazar, at nina.mazar@utoronto.ca.

ELIGIBILITY

At least one author of each presentation must be a member of SJDM. Joining at the time of submission will satisfy this requirement. You may join SJDM at http://www.sjdm.org/join.html. An individual may give only one talk and present only one poster, but may be a co-author on multiple talks and/or posters. Please note that both the membership rule and the one-talk/one-poster rule will be strictly enforced.

NOTE FOR NON-US CITIZENS REQUIRING VISAS

Travelers from certain countries may need extra lead time to obtain travel documents. Although we are unable to accept talks early, we can provide notification of an “accepted presentation.” This means that you would at least be guaranteed a poster. We can do this because posters are typically evaluated only for content and most are accepted. If you submit a talk, you will receive a notice of an accepted presentation immediately, and a decision on your talk at the usual time. To take advantage of this option, you should still submit through the regular process, and also send a request to the program chair, Nina Mazar, at nina.mazar@utoronto.ca.

AWARDS

The Best Student Poster Award is given for the best poster presentation whose first author is a student member of SJDM.

The Hillel Einhorn New Investigator Award is intended to encourage outstanding work by new researchers. Applications are due June 19, 2016. Further details are available at http://www.sjdm.org/awards/einhorn.html. Questions can be directed to Neil Stewart, neil.stewart@warwick.ac.uk.

The Jane Beattie Memorial Fund subsidizes travel to North America for a foreign scholar in pursuits related to judgment and decision research, including attendance at the annual SJDM meeting. Further details will be available at http://www.sjdm.org/awards/beattie.html.

PROGRAM COMMITTEE

Nina Mazar (Chair), Katy Milkman, Suzanne Shu, Ana Franco-Watkins, Thorsten Pachur, Meng Li, Bettina von Helversen, Oleg Urminsky, and Kate Wessels (conference coordinator)