Context: On December 20, 2019, Singular Computing LLC, a research and licensing firm from Massachusetts, filed a patent infringement complaint in its home district against Google, alleging that the latter’s data centers throughout the United States infringed multiple patents on low-precision high-dynamic range (LPHDR) execution units used for AI training. After various delays caused by Google’s partly successful challenges to the validity of the patents-in-suit, a jury trial began earlier this month. The maximum number of what Google might have had to pay exceeded $20B as explained further below. A second Singular Computing v. Google case that had been brought in 2020 went nowhere.
What’s new: Yesterday (Wednesday, January 24, 2024), the trial had essentially been held and a jury verdict could have come down shortly. The parties then asked Chief Judge F. Dennis Sailor of the United States District Court for the District of Massachusetts to stay the case pending a settlement agreement.
Direct impact: The terms are not and likely will never be disclosed, but given what was theoretically at stake and the stage of proceeding, it’s a safe assumption that the payout was substantial. In ai fray‘s estimates, the figure is likely in the many hundreds of millions of dollars, but even a billion-dollar payment cannot be ruled out given the hypothetical financial risk to Google.
Wider ramifications: This is the first major settlement of an AI-related patent infringement action. It’s about chipsets, and AI software patents will presumably face far greater hurdles particularly with a view to patent-eligibility, though legislation is being proposed in U.S. Congress to widen the scope of patent-eligible subject matter, which could result in a substantial number of AI software patent lawsuits in the United States. Even though Singular Computing v. Google was a hardware patent case, Google raised an Alice (§ 101) defense at different stages, and Singular once amended its complaint in response to it.
The claimed invention in a nutshell: sacrifice mathematical precision to get more operations per second
The Massachusetts-based inventor, Dr. Joseph Bates, was a child prodigy. At the age of 13, Johns Hopkins University admitted him to its computer science undergraduate program. By age 17, he already has a master’s degree, and he was 23 when the earned his doctoral degree from Cornell.
In 2009, long before the current AI revolution, Dr. Bates had identified an issue with existing chipsets: they performed relatively few operations per time unit compared to the number of transistors, and one key factor that was slowing them down was the objective of delivering mathematically precise results.
Dr. Bates wanted to create a computing technology that would reprioritize: by accepting a certain degree of imprecision in floating-point (i.e., numbers with decimals) operations, performance would go up, not only gradually but by a huge factor. That would make such chips ineligible for, say, accounting software. You just couldn’t run a bank on such hardware. But for use cases where some degree of imprecision is tolerable or even next to irrelevant, the performance gains are worth it.
Floating-point calculations by binary systems are not 100% precise in all cases because some numbers just can’t be represented in a binary format short of an infinite number of digits. For the average end user who’s not performing scientific work, floating-point imprecision is like an eradicated virus: you don’t encouter it anymore. In an earlier computing era, however, even end users noticed it. For example, the Commodore home computers of the early 1980s would not return 3 for the square root of 9, but a number like 2.99997. With even numbers they were more reliable. Back in the day, when some small business operators used such systems for calculating Value Added Tax or (in the U.S.) sales tax, they could get into trouble with tax authorities because of rounding errors that, in the aggregate of many operations, could add up and potentially result in underpayment.
AI training involves huge numbers of floating-point operations, but the net effect that really matters is the result of a huge number of operations. The result is more likely to be useful if it’s derived from a larger number of somewhat imprecise calculations (of course, within reason) than from a smaller number of more precise ones.
That concept also applies to Graphics Processing Units (GPUs), which are now widely used for AI purposes. If a 3D image is drawn in a high resolution and at a high frame rate, any rounding errors will not be noticeable to the eye. Quite often they won’t even change which pixels have which color, and even if they rarely do, it won’t matter because it’s the overall impression and perception that counts.
Dr. Bates approached tech companies such as Google, Apple, Fujitsu
Documents from the Singular Computing v. Google litigation show that Dr. Bates met with Google in 2017 about its AI applications at the time, such as Google Translate, and there are also some heavily-redacted references to his conversations and “evaluation agreements” (contracts enabling others to assess whether there is a potential for a partnership) with such companies as Apple and Fujitsu.
Singular Computing’s website mostly contains links to scientific writings and patent documents, and says that the company “develops and licenses hardware and software technologies for high performance energy efficient computing, both large scale and embedded.” It explains that those are general-purpose technologies that “support deep learning, others sorts of AI, and non-AI tasks.” The invention (also desribed as “approximate machines”) allegedly “yield[s] the same quality results as modern GPUs, but with 30-50x better compute/watt.”
Ahead of trial, Google agreed not to label Singular Computing a “patent troll” or to use various similar terms. U.S. courts generally disallow such pejorative labels.
The alleged infringement and Google’s invalidity challenges
Somehow Google and Dr. Bates didn’t work it out, and in late 2019, his company Singular Computing LLC brought its first patent infringement complaint. It also brought another one a few months later over a couple of other patents, but the second case went nowhere and was withdrawn.
Singular essentially alleged that Google infringed its patents by deploying across all of its data centers the Tensor chips that are optimized for machine learning.
Google disputed both infringement and validity. One of Singular’s patents-in-suit in this case was invalidated by the Patent Trial and Appeal Board (PTAB) of the United States Patent & Trademark Office (USPTO). But the judge didn’t grant Google’s motions to hold any patents invalid or not infringed, so a couple of claims remained to be asserted in the jury trial:
- claim 53 of U.S. Patent No. 8,407,273 and
- claim 7 of U.S. Patent No. 9,218,156.
Both asserted patents share the same title (“Processing with compact arithmetic processing element”) and are from the same patent family.
Google brought two kinds of invalidity challenges. At different stages of proceeding, Google asked the district court to hold those inventions patent-ineligible under 35 U.S.C. § 101, which is also known (named after a famous precedent) as the Alice defense. That argument is going to come up in virtually every AI patent lawsuit, and is going to result in many of them being thrown out, unless and until United States Congress passes legislation that lowers the hurdle for patent-ineligibility. The latter may happen as there’s significant bipartisan support for a proposal named the Patent Eligibility Restoration Act. It is, in terms of political momentum, the most promising proposal to have been made in U.S. Congress in a long time of the kind that would be designed to benefit only patent holders and inventors seeking patents (as opposed to general reforms or measures such as the America Invents Act, which was actually more of a defendant-friendly piece of legislation).
For the avoidance of doubt, patent-ineligibility is not the same as saying that something is non-novel or obvious over the prior art (earlier publications, which often are, but need not be, other patent applications). Patent-ineligibility under § 101 means that an invention covers an abstract idea. There’s a two-step test. First the question is whether the idea is abstract. If it’s not, then the question is whether the elements based on which the court thinks there’s something concrete there are sufficiently inventive (as opposed to merely boilerplate).
On that basis, Google wanted the original complaint thrown out. When Singular saw Google’s § 101 argument, the first reaction was to optimize the complaint by making it clear (through the allegations contained therein) that Dr. Bates had invented something concrete, a specific way to design more efficient chipsets. The judge denied Google’s motion to dismiss the amended complaint because at that stage it’s only about the pleadings and those were sufficient. Google tried again later on summary judgment (when some facts had been established) and failed again. Singular brought a cross-motion asking the court to hold that Google could not keep pursuing its § 101 defense.
As this litigation has apparently been settled, there won’t be an appeal. Until the Federal Circuit has decided on whether those patents cover purely abstract subject matter are not, anyone being sued by Singular is at risk of losing, but Singular also has the risk that next time the patents could be invalidated. At this stage, given that there may not even be any subsequent lawsuits, ai fray prefers not to opine on whether Google’s § 101 defense was meritorious.
Google additionally challenged the patent claims before the aforementioned PTAB, with the result that two survived and made it to trial, though Google succeeded against another patent. In the PTAB, there is no § 101 argument but the focus is strictly on whether a patented invention meets the other patentability criteria relative to the prior art.
There was also an interesting procedural tactic that Google used. They had effectively waived a certain invalidity theory but then said they couldn’t have presented actual hardware to the PTAB (under USPTO rules), while they could do so in district court, and then they made an invalidity argument based on a combination of physical evidence and documents. The judge made it clear that if it was a 50-50 situation where the physical part was roughly as important as the documents in question, then that would be acceptable, but if the physical evidence was just used for a pretext to revive an otherwise-waived defense, he wouldn’t let Google do that. But that is not an AI-specific question, while § 101 is going to remain a huge topic in connection with AI patent enforcement.
The case was temporarily stayed (from mid 2021 to mid 2022) and Google obviously exhausted all of its procedural options. Singular insinuated at times that Google was stalling, but given that Google’s PTAB challenges were at least partly successful and that the case went to trial after a little over three years despite the one-year stay, it’s fair to say that the court’s case management was efficient and Singular can’t complaint. Now that they’ve received a presumably substantial payment, they obviously won’t complain anymore.
The damages claim
The last aspect of the case to be addressed here is how much Singular was seeking in tihs case and how much they could theoretically have won if the case not settled and the jury and the judge had agreed with them to the maximum extent (and if Google hadn’t achieved anything on appeal, though massive damages verdicts in patent cases typically are deflated a lot or even tossed).
Essentially, Singular’s argument was that Google saved about $10B in data center-related costs by using the patented invention. Singular wanted a nice cut of that figure.
There have been some media reports on the settlement that reference different figures. ai fray focuses on what Google’s trial brief said:
- “Singular is not entitled to its inflated request for reasonable royalty damages of up to $7.01 billion. This staggering amount, given Singular’s expert Philip Green’s opinions, necessarily wouldn’t be tied to the smallest salable patent-practicing unit—i.e., functionality within the TPU chip, rather than the TPU system.”
- With respect to Singular’s lump-sump royalty theory, Google’s trial brief says “Singular seeks damages of no less than a lump sum of between $1.63 and $5.19 billion.” A footnote then explains that “Singular’s damages expert also provides a lump-sum damages range of $3.3 to $7.01 billion,” but Google goes on to argue that “[t]his larger range improperly uses TPU global deployment data instead of domestic deployment data, further inflating the calculations.” Patents are territorial rights, so a U.S. patent only entitles to damages related to infringement in the United States (sometimes non-U.S. actions go into a damages determination, but it would lead too far to discuss such scenarios here).
It’s definitely wrong to say Singular wanted $1.63B, as even the lower one of the two ranges stated in Google’s trial brief etends to $5.19B at the upper end. Given that Google twice mentions the $7.01B figure (no matter how questionable the approach of including non-U.S. data centers may be), it’s actually correct to say that what Singular told the jury was that they should award (up to) more than $7B.
But Singular also wanted the jury to determine that Google (in light of Dr. Bates’ discussions with them) had willfully infringed the patents-in-suit. On that basis, Singular would have sought a damages enhancement (not from the jury, but subsequently from the judge). Such damages enhancements can result in a tripling of the award (“treble damages”), which in this case means that the hypothetical maximum award would have exceeded $20B. Even if we multiply by three just the $5.19B figure, that would have been more than $15B.
The AI era has its first major patent settlement. There’ll be more, and even more so in the event United States Congress adopts the Patent Eligibility Restoration Act, on which ai fray does not take a position here other than predicting a massive increase in AI-related litigation and AI-related patent filings.