Caplan on the autodidact's curse
From his interview with Tyler:
Tyler Cowen: So, say you write an interdisciplinary chapter for one of your books. Tell us a little more of how you do this and how you calibrate what you’re reading against actual reality.
Bryan Caplan: My procedure, which I’ve been pursuing more and more as I go along, is first of all, I start with the big topic. I’m usually using Google Scholar to try and find what has anyone written on this big topic? I go through the first 20 or 30 pages of Google Scholar views, but then that to me just gives you a ballpark of what’s really going on.
Then I try to subdivide every topic into lots of separate subtopics, and then repeat that same process of going through Google Scholar just to see what is it that almost anyone has said about this topic. When I actually get the papers, then I actually will go and look at the references and see if there’s more stuff that I should be looking at here.
I get those papers and go back to Google Scholar, and eventually the process does converge. I do have some stopping rules. I generally don’t worry too much about empirical papers written before 1980.
Cowen: You also then email the people who’ve done this research, right?
...
Caplan: When I’m going through and reading papers, oftentimes I’ll find that there’s something I don’t understand very well or something that seems questionable. Usually, as long as the authors are living, I do try to actually reach out to them and get clarifications.
The next thing is when I’ve got what I think is a good, solid draft of a book. That’s where I enlist my RA and say, “Get me all the emails of all the living people I’ve cited in the book so far,” and then email all of them with two offers.
One of them is an offer to show them the entire manuscript. The other one — so that I’m not over on the far right side of the Laffer curve – is to say, “Or, if you’re busy, then I could just tell you the exact pages where I discuss your works, so at least you can tell me whether I’m accurately summarizing your work or not.”
For me, what I do is so interdisciplinary so I’m always worried about this autodidact’s curse, where you’ve read a ton of stuff but you still haven’t actually talked to anyone who knows what’s going on. This is one of the things that I try to do to deal with especially the wisdom of a field. Oftentimes there’s wisdom in a field, where it’s known to people who have thought about it for a long time, but they don’t write it down.
Of course, that’s very hard for the autodidact to find out. “What is the wisdom in your field that you don’t write down?” This is where I try to reach out to people. Generally, I would say I get about a 15 percent response rate for the people saying they’ll at least read something, so I feel like it does give me some good quality control.