DCT

6:21-cv-00511

Freshub Inc v. Amazon.com Inc

Log In

I. Executive Summary and Procedural Information

  • Parties & Counsel:
  • Case Identification: 1:19-cv-00885, W.D. Tex., 09/18/2019
  • Venue Allegations: Venue is alleged based on Amazon's regular and established places of business in the Western District of Texas, including a large office and multiple fulfillment centers, and its continuous business activities targeting residents of the district.
  • Core Dispute: Plaintiff alleges that Amazon's Alexa-enabled products and the associated Alexa Voice Service infringe four patents related to voice-based capture, processing, and management of shopping lists and product orders.
  • Technical Context: The technology at issue resides in the high-growth market of voice-controlled smart home devices and their integration with e-commerce platforms.
  • Key Procedural History: The complaint alleges a long history of pre-suit knowledge by Amazon, including a 2010 request and receipt by an Amazon employee of a wireless scanner product developed by Plaintiff's predecessor that was marked with related patents. It further alleges Amazon cited a parent to the asserted patents during its own patent prosecution and that the parties engaged in partnership discussions between 2015 and 2019.

Case Timeline

Date Event
2004-12-10 Earliest Priority Date for all Asserted Patents
2010-11-01 Amazon allegedly receives scanner product from Plaintiff's predecessor
2014-01-01 Alleged start date of Amazon targeting products/services to the District
2018-03-06 U.S. Patent No. 9,908,153 Issues
2019-02-26 U.S. Patent No. 10,213,810 Issues
2019-03-19 U.S. Patent No. 10,232,408 Issues
2019-03-26 U.S. Patent No. 10,239,094 Issues
2019-09-18 Amended Complaint Filed

II. Technology and Patent(s)-in-Suit Analysis

U.S. Patent No. 9,908,153 (the "’153 Patent")

  • Patent Identification: U.S. Patent No. 9,908,153, "SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM STORAGE AREA CONTENTS," issued March 6, 2018.

The Invention Explained

  • Problem Addressed: The patent addresses the inconvenience and waste associated with managing perishable items in storage units like refrigerators, where expiration dates are often difficult to monitor, leading to items being kept past their expiration (U.S. Patent No. 10,213,810, col. 1:24-34).
  • The Patented Solution: The invention proposes a distributed system for voice-based shopping. A local "first system" (e.g., a device in the kitchen) captures a user's spoken order with a microphone, digitizes it, and "immediately" transmits the digitized data to a remote computer system. This remote system then performs the heavy lifting: translating the voice data to text, identifying the corresponding product, adding it to a list, and making that list available for display to the user (’153 Patent, Claim 1; U.S. Patent No. 10,213,810, Fig. 8).
  • Technical Importance: The technology aimed to streamline in-home commerce by allowing users to create and manage shopping lists using natural voice commands, offloading the complex voice processing to a more powerful remote server (U.S. Patent No. 10,213,810, col. 14:14-22).

Key Claims at a Glance

  • The complaint asserts independent claim 1 and dependent claims 2-11 (Compl. ¶42).
  • Independent Claim 1 requires a voice processing system comprising:
    • A "first system" with a microphone, wireless network interface, digitizer, computer, and memory.
    • This first system is configured to perform operations including: receiving a verbal order from a user and "immediately" transmitting the digitized order to a remote computer system.
    • A "computer system remote from the first system" with a network interface, computer, and memory.
    • This remote system is configured to perform operations including: receiving the digitized order, translating it to text, identifying the corresponding item, adding the item to a list, and enabling the list to be displayed.

U.S. Patent No. 10,213,810 (the "’810 Patent")

  • Patent Identification: U.S. Patent No. 10,213,810, "SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM STORAGE AREA CONTENTS," issued February 26, 2019.

The Invention Explained

  • Problem Addressed: Similar to the ’153 Patent, the technology addresses the management of household inventory and the creation of shopping lists (’810 Patent, col. 1:24-34).
  • The Patented Solution: This patent describes a cloud-centric system architecture. A central computer system associates a unique identifier with a "remote system" (the voice-capture device) and downloads configuration data to it. The central system receives a digitized voice order from the remote device, processes it (translates to text, identifies the item), adds the item to a "set," and enables that set to be displayed on a "user device different than the remote system" (e.g., a smartphone) and provided to an "item provider" (e.g., an online retailer) (’810 Patent, Claim 1; Fig. 2).
  • Technical Importance: The invention describes a more advanced, cloud-managed ecosystem where a simple, dedicated voice-input device can interact with a powerful backend that syncs shopping lists across multiple different user devices and can initiate transactions with retailers (’810 Patent, col. 8:8-15).

Key Claims at a Glance

  • The complaint asserts independent claims 1 and 17 and dependent claims 2-16 and 18-29 (Compl. ¶62).
  • Independent Claim 1 requires a voice processing system (the central server) comprising a computer and memory configured to perform operations including:
    • Associating a unique identifier with a "remote system" (the voice-capture device).
    • Downloading configuration data to the remote system.
    • Receiving a digitized order from the remote system.
    • Translating the order to text to identify an item.
    • Including the item in a "set of one or more items."
    • Enabling the set to be displayed on a "user device different than the remote system."
    • Enabling the set to be provided to an "item provider."

U.S. Patent No. 10,232,408 (the "’408 Patent")

  • Patent Identification: U.S. Patent No. 10,232,408, "SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM STORAGE AREA CONTENTS," issued March 19, 2019.
  • Technology Synopsis: This patent claims a voice processing system that receives a digitized voice order from a remote device, translates it to text, and critically, matches that text to a text description stored in a database that also contains "associated unique product identifiers." Based on this match, the system identifies the item and adds it to a user's list (’408 Patent, Claim 1). This invention focuses on the specific step of matching spoken text to a structured product database containing unique identifiers like UPCs or ASINs.
  • Asserted Claims: Claims 1-30 are asserted (Compl. ¶93).
  • Accused Features: The complaint alleges that Amazon's AVS infringes by translating voice commands and matching the resulting text against its extensive product catalog, which uses unique identifiers (ASINs), to identify and add items to a user's shopping cart (Compl. ¶104, ¶106).

U.S. Patent No. 10,239,094 (the "’094 Patent")

  • Patent Identification: U.S. Patent No. 10,239,094, "SYSTEMS AND METHODS FOR SCANNING INFORMATION FROM STORAGE AREA CONTENTS," issued March 26, 2019.
  • Technology Synopsis: This patent describes a computer-implemented method where a first computer system (remote server) receives a digitized voice order from a second computer system (local device). A key distinction is that the translation from voice to text is performed by a "translation module executed by the second computer system" (the local device) before the text is matched at the remote server (’094 Patent, Claim 1). This shifts the primary voice-to-text processing from the remote server to the local client device.
  • Asserted Claims: Claims 1-24 are asserted (Compl. ¶122).
  • Accused Features: The complaint alleges the Accused Products perform this method, with the local device translating the spoken order to text before the remote AVS server matches that text to a data store to identify the item (Compl. ¶126).

III. The Accused Instrumentality

Product Identification

The accused instrumentalities are Amazon's "Accused Products," which collectively include Echo smart speakers, Fire TV media players, Fire Tablet computers, and the Amazon and Alexa mobile applications (Compl. ¶28). These products are all integrated with Amazon's Alexa virtual assistant and the backend Alexa Voice Service ("AVS") (Compl. ¶29).

Functionality and Market Context

The Accused Products use microphones to capture a user's voice commands, which are then processed by the cloud-based AVS (Compl. ¶34, ¶38). A core accused functionality is "Shopping with Alexa," where a user can issue commands like "add [product] to my shopping cart" or "add [product] to my list" (Compl. ¶30). AVS processes these commands to convert speech to text, identify the intended product, and add it to the user's Amazon shopping cart or a dedicated "Alexa Shopping List" (Compl. ¶54). The complaint alleges these lists are synchronized and viewable across multiple devices, such as an Echo Show display or the Amazon mobile app, separate from the initial voice-capture device (Compl. ¶32, ¶55). A diagram in the complaint illustrates the data flow from a user's utterance captured by a device to the AVS cloud for Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) processing (Compl. ¶38).

IV. Analysis of Infringement Allegations

’153 Patent Infringement Allegations

Claim Element (from Independent Claim 1) Alleged Infringing Functionality Complaint Citation Patent Citation
a voice processing system comprising: a first system configured to receive user spoken words comprising: a microphone, a wireless network interface, a digitizer... The Amazon Echo, Fire TV, and Fire Tablet products are identified as the "first system," and product specifications are cited to show they include microphones, Wi-Fi connectivity, and components to digitize audio. ¶46-49 col. 14:50-59
receive via the digitizer a verbal order... wherein the verbal order was captured by the microphone and digitized by the digitizer; The AVS client software on the Accused Products receives a user's verbal request (e.g., to add an item to a list), which is captured by the microphone and digitized into a format like PCM. ¶51-52 col. 14:10-12
immediately transmit, using the wireless network interface, the digitized order to a computer system remote from the first system; The device sends the digitized audio via Wi-Fi to the remote AVS cloud component for processing. A provided diagram shows this communication link (Compl. ¶52). ¶52 col. 14:12-13
the computer system... comprising: a networks interface; a second computer... perform operations comprising: receive... the digitized order...; translate... to text; identify an item...; add the identified item to a list... The AVS computers/servers in the cloud receive the audio, use ASR to convert speech to text, and use NLU to identify the item and add it to the user's Alexa Shopping List. ¶53-55 col. 14:14-22
enable the list, including the identified item, to be displayed via a user display. The Alexa Shopping List is viewable on devices with displays, such as an Echo Show or a smartphone running the Amazon App. The complaint includes a screenshot of such a list (Compl. ¶32). ¶55 col. 14:22-24

’810 Patent Infringement Allegations

Claim Element (from Independent Claim 1) Alleged Infringing Functionality Complaint Citation Patent Citation
associate a unique identifier with a remote system configured to receive user spoken words... Amazon's system associates a unique Device Serial Number (DSN) with each Alexa-enabled device, which is also linked to a user's unique Amazon account. ¶71-72 col. 14:38-40
download configuration data to the remote system; User settings for an Alexa device are configured remotely via the Alexa app, and this configuration data is then downloaded to the specific device. ¶77 col. 14:48-49
receive, using the network interface, a digitized order of a user from the remote system; The AVS cloud servers receive the digitized voice command from the user's Echo or other Alexa-enabled device. ¶74-75 col. 14:50-52
translate... to text; use the text... to identify an item...; include the identified item in a set of one or more items associated with the user; AVS uses ASR and NLU to process the order, identify the item, and add it to the user's shopping cart or list. ¶76, ¶78 col. 14:53-59
enable the set of items... to be displayed to the user via a user device different than the remote system; A shopping list created via a voice command to an Echo speaker (the "remote system") can be viewed on a separate smartphone running the Amazon App or on a web browser (the "user device"). ¶78 col. 15:1-4
and enable the set of items... to be provided to an item provider. Items added to the Amazon shopping cart can be ordered and purchased from Amazon, the item provider. ¶78 col. 15:5-7

Identified Points of Contention

  • Technical Questions: A potential dispute may arise over the specific processing steps occurring on the local device versus the remote server. For the '094 patent, which requires translation on the local device, evidence of where voice-to-text processing actually occurs will be critical. For the '153 patent's "immediately transmit" limitation, any significant pre-processing or buffering on the local device before transmission could be a point of non-infringement.
  • Scope Questions: The term "user device different than the remote system" in the ’810 Patent raises a definitional question. If a user issues a command to an Amazon Echo Show (which has a screen) and the shopping list is displayed on that same Echo Show, it is an open question whether the infringement theory holds, as the voice-capture system and the display system would be the same device.

V. Key Claim Terms for Construction

  • The Term: "immediately transmit" (from ’153 Patent, Claim 1)

    • Context and Importance: This term's construction is critical to defining the temporal relationship between the local device's capture of a voice command and its transmission to the remote server. A narrow definition could allow Amazon to argue that its devices perform intermediate steps (e.g., wake-word processing, buffering, pre-filtering) that defeat "immediacy."
    • Intrinsic Evidence for Interpretation:
      • Evidence for a Broader Interpretation: The specification does not provide an explicit definition, which may support an interpretation based on its plain and ordinary meaning, such as "without undue or material delay." The overall architecture described suggests offloading processing to a remote server, which inherently favors rapid transmission.
      • Evidence for a Narrower Interpretation: The flowchart in Figure 8 of the related patents shows the "DIGITIZE AND RECORD SPOKEN ORDER" step (802) is followed directly by "TRANSMIT ORDER TO REMOTE SERVER" (804), with no intervening steps described (’810 Patent, Fig. 8). This may support an argument that the term requires transmission as the very next step after digitization.

  • The Term: "a user device different than the remote system" (from ’810 Patent, Claim 1)

    • Context and Importance: This term is central to the claimed multi-device architecture. Infringement depends on whether the device displaying the shopping list is considered "different" from the device that captured the voice command. Practitioners may focus on this term because Amazon's ecosystem includes both integrated devices (like the Echo Show) and separate devices (like an Echo Dot and a smartphone).
    • Intrinsic Evidence for Interpretation:
      • Evidence for a Broader Interpretation: The claim language distinguishes between the "remote system" (defined by its components: microphone, wireless interface, etc.) and the "user device" where the list is displayed. This suggests they are functionally and conceptually distinct roles, even if they could be housed in the same physical unit. The specification’s Figure 2 shows a "REMOTE SYSTEM" (214) and a "LOCAL COMPUTER" (202) with a screen (206) as distinct blocks, supporting the idea of a distributed architecture.
      • Evidence for a Narrower Interpretation: A plain reading suggests two physically separate devices are required. If a user speaks to and views the list on the same Echo Show, a defendant could argue the "user device" is not "different than" the "remote system," thereby avoiding infringement for that use case. The claim itself does not explicitly state that the two cannot be part of the same housing.

VI. Other Allegations

  • Indirect Infringement: The complaint alleges induced infringement for the ’810, ’408, and ’094 patents (Compl. ¶85, ¶114, ¶142). The basis for this allegation is that Amazon instructs and encourages its customers, via advertising, user manuals, and online help sections, to use the Accused Products in a manner that directly performs the steps of the method claims (Compl. ¶87-88, ¶116-117, ¶144-145).
  • Willful Infringement: Willfulness is alleged for all four asserted patents (Compl. ¶58, ¶81, ¶110, ¶138). The allegations are based on pre-suit knowledge of the patent family, stemming from: (1) Amazon's 2010 receipt of a physical product from Plaintiff's predecessor marked with related patents; (2) Amazon's citation to the ’344 parent patent during its own patent prosecution; and (3) years of alleged partnership discussions between the parties (Compl. ¶57).

VII. Analyst’s Conclusion: Key Questions for the Case

  • A core issue will be one of evidentiary proof: can Plaintiff demonstrate through technical evidence that the internal workings of Amazon's complex, multi-component Alexa Voice Service perform the specific sequence of steps—particularly concerning the location and timing of voice-to-text translation—as recited in the distinct claims of the four asserted patents?
  • A second key question will be one of claim construction: can the term "immediately transmit" from the ’153 patent be interpreted to cover a system that may perform wake-word detection or other pre-processing on a local device, and can "a user device different than the remote system" from the ’810 patent read on an integrated device like the Amazon Echo Show?
  • Finally, the case will likely feature a significant focus on willfulness: given the detailed factual allegations of Amazon's pre-suit knowledge of the technology dating back nearly a decade before the suit, a central question for the factfinder will be whether Amazon's conduct rose to the level of wanton or reckless behavior that would justify enhanced damages.