Law school sorting and the partnership track: Northwestern empiricists weigh in
Some extremely interesting, powerful comments on the Henderson argument from my able empirical-legal colleagues:
UPDATE: I have taken two of the posts which were written anonymously.
From NU Prof. Kate Litvak:
In short, the averages he is discussing cannot tell us anything useful. The main problem is selection effects combined with the level of data aggregation: (1) no individual-level data on hired/promoted lawyers; (2) all top-250 law firms are grouped together; (3) all non-T14 law schools are grouped together.
Let’s see how this matters for his explanations from the blog post:
- Selection effects. Bill’s story: best student from a lower-ranked school is better than a midrange student at T14, and the better associate eventually gets promoted. Maybe, but he didn’t actually show any such thing. For this, he needs data on class standing or other quality measure of young lawyers. More importantly, he needs to compare promotion rates within each law firm. For example, his methodology cannot rule out the following alternative hypothesis: many Biglaw firms (especially smaller ones or with offices in smaller legal markets) use merit-based criteria to hire only T-14 associates, and then, leave some spaces for the kids of important local people, non-T14 hires, who are then promoted on the same non-merit-based grounds. Generally, there are many reasons to hire/promote a person who has less-than-stellar formal credentials; hidden high ability is only one of those reasons.
- First jobs. Bill’s story: T14 grads are more likely to start at firms with lower promotion rates. Probably true, but again: (a) he has to show it by looking at promotion rates within each firm, and (b) if this explanation is correct, his overall results might be entirely spurious. As an illustration: a well-known old study of graduate school admission rates at Berkeley. It found that women’s grad school applications are rejected at a much higher rate than men’s applications, which signaled sexism. But when they looked department-by-department, men and women had identical admissions rates. The reason is: women disproportionally applied to humanities, with high rejection rates, while men applied to hard sciences, with low rejection rates. The “disproportional admissions rate” result was a statistical gaffe, caused by the failure to disaggregate the data by departments. Same is likely with Bill’s results: within each law firm, T14 grads might have significantly higher changes of being promoted; it’s just that T14 grads are more likely to work at firms with low promotion rates. Bill also mentions that T14 grads are less likely to do corporate law; if promotion rates differ depending on the legal field, a proper study must control not only for the law firm, but also for the legal field. This might become problematic due to the small sample size, which is one of many reasons to use longitudinal data, instead of cross-sectional, as this one.
- Inter-generational privilege. Bill’s story: T14 grads are more likely to be independently wealthy and don’t want to work hard. Maybe, but a pure speculation at this stage, and cannot be tested through aggregate data. For example, even if T14 grads are disproportionally wealthier than the rest, it doesn’t mean T14 grads who accepted grueling jobs at Biglaw firms are wealthier – the rich ones might be moving to low-stress or high-prestige non-Biglaw environment immediately after graduation. Even if individual-level data is not available, this is where disaggregating law firms into smaller chunks would really help. A rich grad might well take a job at a top-10 firm just to see what it’s about, but why take a job at a firm #200, especially outside glitzy metropolitan areas?
- Influence of admissions criteria. Bill’s story: T14 grads are “academics” and lack life experiences; life experiences increase one’s chances of partnership. This strikes me as dubious. It’s not obvious that a top grad of a low-ranked school (the one most likely to be hired by Biglaw on the merits) would be the “significant life experience” type. I would expect that person to be among the most academic at their school, and likely more academic than a midrange T14 grad working at the same firm. In any case, this too cannot be tested with aggregate data; simple individual-level data like age might be a decent proxy for life experience. Even if individual-level data is not available, this is where disaggregating non-T14 into smaller chunks would help: a Vanderbilt or USC grad is not likely to have more life experiences than a Georgetown grad, but a grad from tier-4 might.
- A better plan “B”. Bill’s story: T14 grads have better alternatives, so they don’t try hard. Again, maybe, but has to be tested on individual-level data (looking at where Biglaw alums got their next jobs); or at least by looking at rates of high-prestige non-Biglaw jobs for grads of each school and the rates of high-prestige post-Biglaw employment for alums of each law firm; or at the very least, by breaking down both law firms and law schools into smaller chunks for study.
And from NU Prof. Max Schanzenbach:
What to make of Bill’s piece? I don’t think it tells us much. Bill is looking at the conditional probabilities. What is the chance making partner CONDITIONAL on being hired as an associate. If firms are willing to take fewer risks on Loyola students (e.g., we hire the number one at Loyola and that’s it), then it’s not surprising that the conditional probability is higher. In my view, this result likely suggests a bias in favor of T14 students (perhaps for rational reasons regarding search costs), and an even stronger reason to attend an elite school if one’s goal is to be a top 250 law firm partner.
Comments (non-anonymous welcome)