You may have heard recently about how an obscure graduate student from UMass debunked a very influential paper from very influential professors at very influential Harvard recently. Indeed, it was a takedown.
Empirical economists Carmen Reinhart and Kenneth Rogoff’s oft-cited paper “showed” that governments reaching or exceeding a debt-to-GDP figure of 90% experience comparatively much lower growth. Turns out the whole thing was riddled with errors. I happen to agree that bad things are coming to countries that over-leverage themselves, but there is simply no way that a study like this was going to be correct, from its very inception.
Put aside for the moment the fact that the word austerity seems to have no useful meaning outside of rigid ideological parameters, meaning those who care more about politics than reality (read: 99% of economists) are talking at cross purposes. The real problem here is methodological. There is simply no way a spreadsheet of math problems based on aggregations of trillions of data points can tell you anything useful about whether a very general fiscal policy, with no set definition anyway, will have particular effects. It is all fantasy.
The most important point, though, is that we learn the right lesson from this. On the one hand, fancy math does not have the ability to tell us whether austerity, as a general matter, is correct. Anyone who thinks so is probably an idiot.
On the other hand, anyone who thinks that errors in fancy math prove the opposite position, is an even bigger idiot.
This is because they make a compounding error – on top of assuming that the methodology could give us useful evidence if it didn’t have its errors, they also assume that the absence of this evidence is evidence of their opposing position. This is a logical fallacy: absence of evidence for position A will never be evidence for position B. (If you’re interested in seeing a prime example of such massive idiocy, feel free to click on this link. I warn you, it isn’t for those with fragile stomachs.)
Ultimately, it is critical not to get caught up in the economic flame war without evaluating first principles. Neither side is right. And neither side seems to know why. In the meantime, I suppose we could just stand back and enjoy the show.
TaxProfBlog highlights a bit of interesting legal scholarship, called “Predicting Lawyer Effectiveness – A New Assessment for Use in Law School Admission Decisions.” Briefly, the paper looks to overcome some difficulties with the Law School Admissions Test by replacing it with psychological measuring for those things which correlate with effective lawyering:
Innovative exploratory research by two UC Berkeley faculty (Marjorie Shultz, Law and Sheldon Zedeck, Psychology) has demonstrated that on-the-job professional effectiveness of lawyers can be predicted. The new Shultz-Zedeck tests, developed based on models from employment selection and promotion and the field of industrial psychology, identify and assess many factors not measured by the LSAT that are vital to lawyer efficacy, such as problem solving, advocacy, practical judgment, and communication skills. Exploratory research conducted with participation of more than 5000 law grads suggests that tests can be developed and validated that will predict professional performance (as appraised by peers and supervisors).
As an attorney myself, I have a couple of problems with this.
The first one is easily the most serious, and I think it highlights the real and stark disconnect between the scholarly pursuits of legal academia and the realities of the real world. Of course, these peer and supervisor reviews sound perfectly good in theory, but as anyone who has submitted to an annual performance review in a company’s down year will know, there is often no true connection between performance and the results of your performance review.
The idea that assessments can be done in an objective manner exhibits, I believe, a willful ignorance of the actual psychology of the workplace. Odd for a psychological study. I would like to believe that perhaps these effects would be tempered by an aggregation study, but even that would matter but little. After all, job performance is an individual metric, as is the decision whether to admit a potential law student.
I simply do not believe that there is a way to remove office politics, which are often backstabbing and brutal. There is no good way to control for the subjective measure of attractiveness, shown to be correlated with increased earnings on average, and more than likely correlated with friendlier performance reviews. There is no good way to control for the questionable effects of height, also shown to be correlated with greater success. And how would one correct for the fact (fact!) that some groups of peers and supervisors are far better than others? After all, being judged effective at a top New York City law firm is different than being judged effective in your job at the local suburban ambulance chaser’s firm. The worst lawyers at the best firms may well be better than the best lawyers at poor firms.
Also, as someone who has studied and taught the LSAT, I refuse to believe that there is anything peculiar to it that disadvantages underrepresented minorities. Anyone who knows the test knows that LSAC bends over backwards to reduce test bias, to the point where the only credible way to posit bias would be to claim that it is tilted against “overrepresented” groups (one might call this “reverse-bias,” but like “reverse-racism,” I revile this term; racism and bias are what they are, and they require no direction, forward or reverse).
Perhaps instead of introducing pseudo-scientific and easily manipulable psychological data points as a replacement for a very cut-and-dry test, we should recognize that the purported lack of qualified minority candidates for law school is far more likely due to our failures in public education, and the massive amounts of structural waste in the public school systems? In reality, it does unqualified minorities no favors to introduce them to a situation where everyone else (whose grades form the curve!) is better equipped to succeed. This leads to underachievement, poor job prospects, usually crushing debt, and ultimately – although not for everyone – perverse consequences.
For the foreseeable future, there will be no perfect way to determine whether law students will succeed as lawyers, nor is there any particular reason to require them to be, given that there are basically no legal job openings but for the very top candidates anyway. I cannot imagine that this psychological profiling program would be successful. And perhaps more importantly, I cannot imagine that we could comprehend ex post what the possibly horrifying unintended consequences would be.