DCT
3:23-cv-02921
Zentian Ltd v. Apple Inc
I. Executive Summary and Procedural Information
- Parties & Counsel:
- Plaintiff: Zentian Ltd. (United Kingdom)
- Defendant: Apple Inc. (California)
- Plaintiff’s Counsel: Bracewell LLP
- Case Identification: 6:22-cv-0122, W.D. Tex., 02/02/2022
- Venue Allegations: Plaintiff alleges venue is proper in the Western District of Texas based on Defendant’s regular and established place of business in Austin, which includes offices with over 7,000 employees and facilities responsible for manufacturing the Mac Pro, one of the accused products.
- Core Dispute: Plaintiff alleges that Defendant’s on-device speech recognition features, including the “Hey Siri” voice trigger, infringe five U.S. patents related to specialized hardware circuits and methods for improving the speed and power efficiency of speech processing.
- Technical Context: The technology concerns hardware-based acceleration for speech recognition, a key technical challenge for enabling fast, low-power, and accurate voice-activated assistants on mobile and power-constrained devices.
- Key Procedural History: The complaint alleges that Plaintiff met with Defendant in 2006 to present its hardware-based speech recognition technology and again in early 2015 with Defendant's Patent Acquisitions department to discuss the patent families at issue. These alleged meetings form the basis of Plaintiff’s willful infringement claims.
Case Timeline
| Date | Event |
|---|---|
| 2002-02-04 | Earliest Priority Date for ’319 and ’140 Patents |
| 2004-09-14 | Earliest Priority Date for ’277, ’377, and ’789 Patents |
| 2006-01-01 | Zentian alleges meeting with Apple’s Silicon Engineering Group |
| 2009-09-08 | U.S. Patent No. 7,587,319 Issues |
| 2011-01-01 | Apple introduces Siri |
| 2011-07-12 | U.S. Patent No. 7,979,277 Issues |
| 2014-09-01 | Apple introduces the “Hey Siri” feature |
| 2015-01-01 | Zentian alleges meeting with Apple’s Patent Acquisitions department |
| 2018-08-28 | U.S. Patent No. 10,062,377 Issues |
| 2020-11-17 | U.S. Patent No. 10,839,789 Issues |
| 2021-04-06 | U.S. Patent No. 10,971,140 Issues |
| 2022-02-02 | Complaint Filed |
II. Technology and Patent(s)-in-Suit Analysis
U.S. Patent No. 7,979,277 - "Speech Recognition Circuit and Method"
The Invention Explained
- Problem Addressed: The patent addresses the shortcomings of conventional software-based speech recognition systems on mobile electronic devices, which were ill-equipped to perform tasks in a fast and power-efficient manner, particularly in noisy environments (Compl. ¶5; ’277 Patent, col. 1:24-49).
- The Patented Solution: The invention proposes a hardware-based speech recognition architecture where distinct functional stages—an audio front end, a distance calculation engine, and a search stage—are connected to enable a "pipelined" data flow (’277 Patent, Abstract). This pipelined approach allows different stages of the recognition process to operate concurrently on different frames of audio data, improving overall throughput and efficiency (’277 Patent, col. 6:50-65; Fig. 18).
- Technical Importance: This hardware-centric, pipelined architecture was designed to enable faster, more power-efficient on-device processing, facilitating features like always-on voice triggers in mobile devices (Compl. ¶¶6, 37).
Key Claims at a Glance
- The complaint asserts independent claims 14, 15, and 16 (Compl. ¶81). Claim 14 is representative:
- Claim 14 Elements:
- A speech recognition circuit, comprising:
- an audio front end for calculating a feature vector from an audio signal...;
- calculating means for calculating a distance indicating the similarity between a feature vector and a predetermined acoustic state of an acoustic model; and
- a search stage for using said calculated distances to identify words within a lexical tree...;
- wherein said audio front end, said calculating means, and said search stage are connected to each other to enable pipelined data flow.
- The complaint reserves the right to assert additional claims (Compl. ¶115).
U.S. Patent No. 7,587,319 - "Speech Recognition Circuit Using Parallel Processors"
The Invention Explained
- Problem Addressed: The patent targets the "computationally intensive search process" that attempts to find the most likely set of spoken words from a given lexicon, which was a significant performance bottleneck in prior art systems (Compl. ¶51).
- The Patented Solution: The invention discloses a hardware architecture that distributes the word recognition task across a plurality of processors connected in parallel (’319 Patent, Abstract). These processors are arranged in groups, with each group connected to a lexical memory, and are controlled by a control processor to process speech parameters in parallel using data from their respective memories (’319 Patent, col. 10:9-20).
- Technical Importance: This parallel processing architecture provides for improved and faster processing, enabling real-time recognition of more complex and robust speech models on power-constrained devices (Compl. ¶51).
Key Claims at a Glance
- The complaint asserts independent claim 46 (Compl. ¶123).
- Claim 46 Elements:
- A speech recognition circuit, comprising:
- an input buffer receiving processed speech parameters;
- a plurality of lexical memories containing in combination complete lexical data for word recognition...;
- a plurality of processors connected in parallel to said input buffer for processing the speech parameters in parallel, said processors being arranged in groups of processors, each group of processors being connected to a lexical memory;
- a control processor controlling each processor to process said speech parameters using partial lexical data read from a respective said lexical memory; and
- a results memory storing the results of the processing... from said processors.
- The complaint reserves the right to assert additional claims (Compl. ¶147).
U.S. Patent No. 10,971,140 - "Speech Recognition Circuit Using Parallel Processors"
- Technology Synopsis: Belonging to the same family as the ’319 Patent, this patent claims a speech recognition circuit having one or more clusters of processors and an acoustic model memory. The circuit is configured to generate an initial score for an audio sample and then use that score to determine whether to continue processing with a larger amount of model data to determine a final score, aiming to enhance power consumption and efficiency (Compl. ¶56).
- Asserted Claims: At least independent claim 1 (Compl. ¶155).
- Accused Features: The accused functionality is Apple's two-tiered processing system, where a low-power "Always On Processor" (AOP) running a small acoustic model (DNN) generates an initial score for the "Hey Siri" phrase, which then determines whether to wake a main processor (e.g., Neural Engine) to run a larger, more accurate DNN for final score computation (Compl. ¶¶165-166).
U.S. Patent No. 10,062,377 - "Distributed Pipelined Parallel Speech Recognition System"
- Technology Synopsis: Belonging to the same family as the ’277 Patent, this patent claims a system comprising distinct programmable devices. The system includes a first device to calculate a feature vector, a second device to calculate distances between the vector and an acoustic model, and a third device to identify words using those distances, along with a search stage (Compl. ¶61).
- Asserted Claims: At least independent claim 1 and dependent claims 2-6 (Compl. ¶184).
- Accused Features: The complaint alleges that Apple's products use a multi-device system, mapping the first programmable device to the "Always On Processor" (AOP), the second to a more powerful processor like the Neural Engine or GPU, and the third to another processor, such as a main application CPU core (Compl. ¶¶189-201).
U.S. Patent No. 10,839,789 - "Speech Recognition Circuit and Method"
- Technology Synopsis: Also in the ’277 Patent family, this patent claims an "acoustic coprocessor" for processing audio signal data. The coprocessor comprises a first interface for receiving a feature vector, a calculating apparatus for calculating distances against an acoustic model, and a second interface for sending the calculated distances (Compl. ¶66).
- Asserted Claims: At least independent claim 10 (Compl. ¶223).
- Accused Features: The complaint accuses the Intel Gaussian & Neural Accelerator (GNA) found in certain MacBook Pro models of being an infringing acoustic coprocessor, alleging it has interfaces to receive feature vectors from a DSP and a compute core for calculating acoustic model likelihoods (Compl. ¶¶224-228).
III. The Accused Instrumentality
Product Identification
- Apple's iPhone, iPad, Watch, AirPods, HomePod, HomePod Mini, Apple TV, and Mac computers (collectively, "Accused Products") (Compl. ¶14).
Functionality and Market Context
- The allegations focus on on-device speech recognition functionalities, primarily the "Hey Siri" wake phrase detection, dictation of text, and on-device processing of Siri user requests (Compl. ¶73). The complaint describes a hardware architecture where a low-power "Always On Processor (AOP)" continuously listens for the wake phrase and, upon detection, activates a more powerful main processor (such as a Neural Engine, GPU, or CPU) to perform more computationally intensive analysis (Compl. ¶¶88, 95, 134-135). The complaint provides a diagram from an Apple technical paper illustrating the initial processing on a low-compute, "always on" processor before passing data to a more accurate main processor (Compl. ¶88, p. 21). Plaintiff alleges Siri is a "core feature" central to the user experience, fulfilling over 25 billion requests per month (Compl. ¶70).
IV. Analysis of Infringement Allegations
’277 Patent Infringement Allegations
| Claim Element (from Independent Claim 14) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| an audio front end for calculating a feature vector from an audio signal, wherein the feature vector comprises a plurality of extracted and/or derived quantities from said audio signal during a defined audio time frame | The "Always On Processor (AOP)" in the Accused Products performs MFCC (Mel Frequency Cepstrum Coefficient) computations on the microphone signal to calculate feature vectors from the audio input. | ¶¶86-88 | col. 5:63-65 |
| calculating means for calculating a distance indicating the similarity between a feature vector and a predetermined acoustic state of an acoustic model | The Accused Products contain a Deep Neural Network (DNN) acoustic model, which runs on a main processor (e.g., GPU or Neural Engine), that calculates probability scores indicating the similarity between the feature vector and predetermined acoustic states. | ¶¶90-95 | col. 6:2-4 |
| a search stage for using said calculated distances to identify words within a lexical tree, the lexical tree comprising a model of words | The Accused Products employ a search stage that uses the calculated distances from the DNN to identify spoken words, such as the "Hey Siri" phrase, based on a language-specific phonetic model. | ¶¶96-98 | col. 6:5-10 |
| wherein said audio front end, said calculating means, and said search stage are connected to each other to enable pipelined data flow | Data allegedly flows in a pipelined manner from the AOP (feature vector extraction) to a second processor like a Neural Engine or GPU (calculating means), and then to the search stage. The complaint provides a speech recognizer diagram from Apple's technical paper to illustrate this data flow (Compl. ¶104, p. 27). | ¶¶103-104 | col. 6:50-54 |
’319 Patent Infringement Allegations
| Claim Element (from Independent Claim 46) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| an input buffer receiving processed speech parameters | The complaint identifies the "Frame Buffer" within the AOP as an input buffer that stores processed speech parameters (feature vectors) for use by the "main processor." | ¶127 | col. 2:20-22 |
| a plurality of lexical memories containing in combination complete lexical data for word recognition, each lexical memory containing part of said complete lexical data | The Accused Products contain FLASH memory and a cache hierarchy (L2, L3 caches), which allegedly function as a plurality of lexical memories storing lexical data for word recognition. | ¶¶128-130 | col. 2:23-28 |
| a plurality of processors connected in parallel to said input buffer for processing the speech parameters in parallel, said processors being arranged in groups of processors, each group of processors being connected to a lexical memory | The main application processors in Apple's SoCs (e.g., A14/A15) are allegedly arranged in clusters (groups) of multiple processor cores. These clusters are connected to the cache memory (lexical memory) and receive data from the "Frame Buffer" (input buffer). The complaint provides die photos of Apple's A14 and A15 chips to illustrate these clusters (Compl. ¶131, p. 34). | ¶¶131-133 | col. 2:29-38 |
| a control processor controlling each processor to process said speech parameters using partial lexical data read from a respective said lexical memory | The AOP is alleged to function as a control processor that detects a potential trigger phrase using a small acoustic model. When a score threshold is exceeded, the AOP wakes up the main processor to analyze the signal using a larger model, thereby controlling it to process speech parameters using lexical data. | ¶¶134-135 | col. 2:39-43 |
| a results memory storing the results of the processing of the speech parameters from said processors | The Accused Products contain SDRAM, which allegedly serves as the main working memory to store the results of the processed speech parameters from the AOP and main processors. | ¶¶136-137 | col. 2:44-48 |
- Identified Points of Contention:
- Scope Questions: For the ’319 Patent, a key question may be whether Apple’s AOP, which performs initial listening and wake-up functions, can be properly characterized as the claimed "control processor controlling each processor." Similarly, the analysis will question whether the combination of on-chip FLASH, various levels of cache, and system DRAM in a modern SoC maps to the claimed "plurality of lexical memories."
- Technical Questions: For the ’277 Patent, the central technical question will be whether the data handoff between Apple’s AOP, its Neural Engine/GPU, and its CPU cores constitutes the specific "pipelined data flow" architecture required by the claim, or if the interaction is functionally different from what the patent discloses.
V. Key Claim Terms for Construction
- The Term: “pipelined data flow” (’277 Patent, Claim 14)
- Context and Importance: This term is central to the infringement theory for the ’277 Patent, which alleges a specific hardware architecture. The definition will determine whether the sequential but distinct processing stages in Apple's devices (low-power AOP -> high-power main processor) fall within the claim's scope.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The specification describes pipelining as a method where different stages of the recognition process can operate concurrently on different frames of data, improving throughput (’277 Patent, col. 6:50-65). Plaintiff may argue that any system achieving this functional outcome, regardless of the specific hardware implementation, exhibits a "pipelined data flow."
- Evidence for a Narrower Interpretation: The patent figures illustrate the pipeline with distinct functional blocks for a "Front End," "Distance Calculation," and "Search Stage" designed for a hardware accelerator architecture (’277 Patent, Fig. 16). Defendant may argue the term is limited by these embodiments to a more formally structured, hardware-defined pipeline rather than the more flexible software-managed data flow between general and special-purpose processors on an SoC.
- The Term: “control processor” (’319 Patent, Claim 46)
- Context and Importance: Plaintiff’s infringement case for the ’319 Patent maps this claim element to Apple's low-power Always-On Processor (AOP). The viability of this theory depends on whether the AOP’s function—detecting a potential wake word and activating the main processor—constitutes "controlling" the main processors as contemplated by the patent.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The patent describes a controller for managing the lexical tree processors (’319 Patent, col. 10:18-20). Plaintiff may argue that the AOP's act of gating and triggering the main processor is a form of control sufficient to meet this limitation.
- Evidence for a Narrower Interpretation: The patent's architecture depicts a central "Search Controller" that appears to actively manage the distribution of tasks to various parallel processors (’319 Patent, Fig. 2, item 27). Defendant may argue that the AOP is merely a preliminary filter or gatekeeper, not a "control processor" that actively directs the subsequent processing tasks of the main CPUs in the manner disclosed in the patent's embodiments.
VI. Other Allegations
- Indirect Infringement: The complaint alleges that Apple induces infringement by providing directions, instruction manuals, and guides that encourage and facilitate the use of the accused speech recognition features by end-users (Compl. ¶¶107, 139).
- Willful Infringement: The complaint alleges willful infringement based on Defendant's alleged pre-suit knowledge of the patents. Plaintiff claims it met with representatives from Apple’s Silicon Engineering Group in 2006 and its patent acquisition department in 2015, where it allegedly discussed its patented technology (Compl. ¶¶74-75, 111, 143).
VII. Analyst’s Conclusion: Key Questions for the Case
- A core issue will be one of architectural mapping: can the distinct hardware elements recited in claims drafted in the early 2000s—such as "calculating means," a "control processor," and a "plurality of lexical memories"—be mapped onto the highly integrated and functionally distributed components of Apple’s modern System-on-Chip (SoC) architecture, which includes an AOP, a Neural Engine, CPU clusters, and unified memory?
- A key question of claim scope will be whether the term “pipelined data flow” can be construed to read on the software-managed handoff of data between a low-power pre-processor and a high-power main processor, or if it is limited to the more formal, hardware-centric accelerator architecture disclosed in the patent’s specification.
- The case may also turn on an issue of technological evolution: does Apple’s implementation of on-device speech recognition represent a system that is fundamentally different in operation from what was patented, or is it merely an efficient, modern-day embodiment of the parallel and pipelined hardware acceleration concepts pioneered by the patents-in-suit?