Hiding the Patent Troll Problem with Statistics

The Government Accountability Office (GAO) just released its long-awaited study on the consequences of patent litigation by NPEs, and we’ve been hearing story after story reporting that the report shows that there is no patent troll problem. (See here and here for examples.)

It just ain’t so. The author left out a lot of patent trolls in his counting, but even with the GAO’s narrow definition, the report says that number of companies sued by patent trolls went up 400% from 2007 to 2011!

If you doubt my statement, you can look at the bottom of page 18 of the report, where it says:

the estimated total number of defendants sued by PMEs more than tripled from 834 in 2007 to 3,401 in 2011…

PME stands for “Patent Monetization Entity,” which is a nice way of saying “patent troll.” And the number of defendants actually more than quadrupled (834 x 4 = 3,336).

The Warren Buffett Walks Into a Bar Problem

If you choose your statistics carefully, it can be easy to shift the narrative. For example, suppose I tell you that the average net worth of every patron in a particular bar went up by billions of dollars in a single day. It sure sounds like everyone in that bar is better off than they were the day before.

But if I then tell you that Warren Buffett walked into the bar, you should realize that nothing changed for any of those bar patrons. Buffett’s net worth is $53.5 billion, which is what made the average net worth skyrocket. But the other people in the bar are exactly as well off as they were the day before.

The GAO report has a similar issue. For some reason, the author chose a headline statistic that makes it look like there isn’t a patent troll problem:

Operating companies brought most of the patent infringement lawsuits from 2007 to 2011. According to our analysis of data for this period, operating companies and related entities brought an estimated 68 percent of all lawsuits. PMEs and likely PMEs brought 19 percent of the lawsuits. PMEs and likely PMEs brought 17 percent of all lawsuits in 2007 and 24 percent in 2011, although this increase was not statistically significant. In contrast, operating companies and related entities filed 76 percent of the lawsuits in 2007 and 59 percent in 2011, a statistically significant decrease.

Except that the percentage of total lawsuits filed doesn’t tell us anything about the scope of the patent troll problem. The report says that the percentage of patent lawsuits attributable to PMEs went from 17% in 2007 to 22% in 2011, which isn’t statistically significant. That doesn’t sound like much at all. Here’s the pie chart from the GAO report:

It sure looks like the brown pie slice, which is PME lawsuits, has stayed pretty much the same. But this figure actually tells you nothing about the number of PME lawsuits filed. It’s the “Buffett” problem.

Here’s the same set of pie charts, adjusted in size to show the number of patent infringement lawsuits filed for each year (in making the next 2 charts I used data graciously provided to me by Mark Lemley; he sent me the total number of patent infringement suits filed by year according to the Lex Machina database):

If the last pie is bigger than the first one, that “PME” slice is going to be bigger. Here’s just the “PME” slices:

These represent the number of PME lawsuits filed from year to year. You can clearly see that the number of PME lawsuits went up quite a bit from 2007 to 2011. And again, this is using the GAO’s interpretation, which left out a number of patent trolls who were included in the academic study of the same data.

That’s also just the number of lawsuits. Because PMEs used to be able to sue multiple companies in a single lawsuit, the number of companies affected is actually much worse. As I noted above, the number of companies actually sued by PMEs quadrupled from 2007 to 2011.

Undercounting the Trolls

There was a paper in 2011 by Sara Jeruss, Robin Feldman, and Joshua Walker that used the same data set as the GAO but came to a very different conclusion. The Jeruss paper found that PMEs went from filing 22% of patent infringement suits in 2007 to 40% of patent infringement suits in 2011.

Why the difference?

Well, these researchers explained their decisions this way:

In order to properly understand the data, however, it is best to aggregate some of the categories. For example, we set an extremely high bar for classifying entities as either operating companies or patent monetization entities by requiring either an entity’s self-classification or a statement in a verifiable court record. However, even with those classified as suspected patent monetization entities or suspected operating companies, we still found ample secondary evidence of their proper categorization. As a result, we believe that operating companies and patent monetization entities should be aggregated with their suspected counterparts.

In addition, our study suggests that individuals and trusts frequently behave like patent monetizers. This should come as no surprise. Modern patent trolling began with small entities and individuals asserting patents against larger companies that provide products or services. In short, parties that we designated as confirmed patent monetization entities are cut from the same cloth as individuals or trusts asserting patent rights, and we concluded that they should be grouped together for this analysis.

In short, we believe that the most appropriate grouping would be to combine confirmed patent monetization entities with suspected monetization entities and individuals/trusts. Looking at this group, the number of lawsuits filed by monetizers as a whole increases greatly, both in absolute terms and as a percentage of all patent infringement cases. Specifically, lawsuits filed by monetizers rose from 22 percent in 2007 to almost 40 percent in 2011…

In other words, if parties acted like trolls, they should be counted as trolls. The GAO report did mention the difference, in a footnote:

The evidence that Lex Machina used to classify an entity as a PME or likely PME and that we then used to review Lex Machina’s classifications is described in appendix I. Another paper using data from Lex Machina presented different proportions of patent monetizing plaintiffs, and these differences may be due to differences in methodology. For example, this study included other plaintiff groups as patent monetizers, including individuals and trusts. See Robin Feldman, Tom Ewing, and Sara Jeruss, The AIA 500 Expanded: The Effects of Patent Monetization Entities, UCLA Journal of Law & Technology (forthcoming).

Considering that those “different proportions” give a completely different picture, it might have been nice to have a little explanation for why the author used a different approach.

The fact is that there is a large consensus that there is a real patent troll problem. (See here, here, and here.) The data in the GAO report adds evidence that patent trolls are a growing problem, despite the author’s narrow definition of “PME” and despite the report burying the fact that the number of companies sued by patent trolls quadrupled from 2007 to 2011.

But, in the short term, we’ll continue knocking down the narrative that the GAO said there was no patent troll problem.