DCT
3:20-cv-06262
NantWorks LLC v. Niantic Inc
I. Executive Summary and Procedural Information
- Parties & Counsel:
- Plaintiff: NantWorks, LLC and Nant Holdings IP, LLC (Delaware)
- Defendant: Niantic, Inc. (Delaware)
- Plaintiff’s Counsel: Diamond McCarthy LLP
- Case Identification: 3:20-cv-06262, N.D. Cal., 02/22/2021
- Venue Allegations: Plaintiff alleges venue is proper in the Northern District of California because Defendant Niantic maintains its principal place of business in San Francisco, regularly conducts business in the district, and has committed alleged acts of infringement there.
- Core Dispute: Plaintiff alleges that Defendant’s augmented reality mobile games, including Pokémon Go and Harry Potter: Wizards Unite, infringe patents related to systems for managing the display of virtual objects based on environmental context and location.
- Technical Context: The dispute is in the domain of augmented reality (AR) for mobile devices, a technology that overlays computer-generated images onto a user's view of the real world, which has significant application in the entertainment and gaming markets.
- Key Procedural History: The complaint notes that during the prosecution of one of the asserted patents, the patent examiner distinguished the claims from prior art by finding the prior art lacked features for determining whether to alter an AR object's presence based on device location and a virtual element attribute. This prosecution history may inform the court’s future claim construction.
Case Timeline
| Date | Event |
|---|---|
| 2011-04-08 | ’051 Patent Priority Date |
| 2013-10-17 | ’518 Patent Priority Date |
| 2016-07-01 | Accused Product Pokémon Go App Launch Date (approx.) |
| 2017-12-01 | Accused Product Dynamic Weather Feature Launch Date (approx.) |
| 2019-06-01 | Accused Product Harry Potter App Launch Date (approx.) |
| 2019-09-03 | ’051 Patent Issue Date |
| 2020-05-26 | ’518 Patent Issue Date |
| 2021-02-22 | Third Amended Complaint Filing Date |
II. Technology and Patent(s)-in-Suit Analysis
U.S. Patent No. 10,403,051 - Interference based augmented reality hosting platforms
The Invention Explained
- Problem Addressed: The patent’s background section describes prior art AR systems as failing "to appreciate that objects within an environment or scene can interfere with each other to give rise to an augmented reality experience" (Compl. ¶23; ’051 Patent, col. 2:26-29). It notes that in prior systems, users often had to manually select between different AR layers, and virtual objects could not realistically interact with or appear behind other objects (Compl. ¶¶22, 24; ’051 Patent, col. 1:53-61).
- The Patented Solution: The invention is an AR platform that uses the concept of "interference" to manage how AR objects are displayed. Based on a device's location and other contextual data, the system determines how elements in a scene "interfere" to either enhance ("constructive interference") or suppress ("destructive interference") the presence of a virtual object, creating a more seamless user experience (Compl. ¶¶25-26; ’051 Patent, Abstract, col. 3:49-52). Figure 4 illustrates this concept, showing an AR object with an enhanced presence from constructive interference versus a suppressed presence from destructive interference (’051 Patent, Fig. 4).
- Technical Importance: The technology claims to provide a more natural and realistic AR experience by moving beyond simple overlays and enabling virtual objects to interact dynamically with their environment (Compl. ¶22; ’051 Patent, col. 1:63-67).
Key Claims at a Glance
- The complaint asserts infringement of at least Claim 1 (Compl. ¶68).
- Independent Claim 1 of the ’051 Patent recites:
- An AR platform system comprising an AR object repository and an AR server.
- The server is configured to obtain digital data from an AR mobile device, including its location and a virtual element attribute.
- It determines at least one context based on the device location and environment.
- It identifies relevant AR objects from the repository corresponding to the context.
- It determines whether to alter the presence of a relevant AR object based on the device location and the virtual element attribute.
- It causes the mobile device to render the AR object according to its altered presence.
- The complaint anticipates that additional claims may be infringed (Compl. ¶88).
U.S. Patent No. 10,664,518 - Wide Area Augmented Reality Location-Based Services
The Invention Explained
- Problem Addressed: The patent addresses the problem that prior location-based AR systems "fail to consider that areas have various views of interest, and fail to differentiate between sub-areas based on AR content densities" (Compl. ¶38; ’518 Patent, col. 2:1-5). Existing systems did not effectively segment large areas into clusters for delivering relevant AR content (Compl. ¶43; ’518 Patent, col. 2:5-8).
- The Patented Solution: The invention describes a device that provides AR content using a "tessellated" area tile map. The device identifies a specific "tile subarea" based on the device's precise location and accesses AR content objects associated with that tile. This allows the system to efficiently populate the device's memory with relevant, location-specific AR content for rendering to the user (Compl. ¶39; ’518 Patent, col. 4:48-52).
- Technical Importance: This method provides an improved technological framework for delivering location-based AR services over wide areas by organizing AR content into a manageable, tile-based structure (Compl. ¶37; ’518 Patent, col. 1:15-16).
Key Claims at a Glance
- The complaint asserts infringement of at least Claim 1 (Compl. ¶93).
- Independent Claim 1 of the ’518 Patent recites:
- A device capable of rendering AR, comprising a sensor, display, memory, and processor.
- The device is configured to obtain sensor data including a device location.
- It obtains an area of interest from a database based on the location.
- It accesses an "area tile map" for that area, which is represented by a set of "tile subareas" that includes "tessellated tiles."
- It identifies a specific tile subarea based on the device's location relative to the tile subareas.
- The identified tile subarea has one or more tessellated tiles associated with AR content objects.
- It populates the device memory with the AR content objects associated with the identified subarea's tiles.
- It renders the AR content objects on the display.
- The complaint anticipates that additional claims may be infringed (Compl. ¶109).
III. The Accused Instrumentality
Product Identification
- The accused instrumentalities are Niantic’s Pokémon Go App and Harry Potter App ("Accused Products"), operating on Android and iOS mobile devices in conjunction with the backend "Niantic Real World Platform" (Compl. ¶¶4, 52, 55).
Functionality and Market Context
- The Accused Products are location-based AR games that use a mobile device's camera and GPS to superimpose virtual characters (AR objects) onto a digital representation of the user's surroundings (Compl. ¶52).
- The system architecture relies on a backend platform comprising servers, an "AR Cloud," and an "AR DATA" repository, which interact with the client application on the user's device (Compl. ¶¶54, 71). The complaint provides a schematic of the Niantic Real World Platform showing client-server interaction for game functions and AR data handling (Compl. ¶54, p. 19).
- A "Dynamic Weather" feature is alleged to adjust the in-game AR environment based on real-world weather conditions near the mobile device (Compl. ¶53).
- The complaint alleges the Pokémon Go App is a commercially significant product, having generated $3.6 billion in its first four years from approximately 105 million users in the United States (Compl. ¶47).
IV. Analysis of Infringement Allegations
’051 Patent Infringement Allegations
| Claim Element (from Independent Claim 1) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| an AR object repository storing available AR objects in a first non-transitory computer readable memory | The Niantic Real World Platform includes stored "AR DATA," which allegedly functions as a repository of available AR objects (e.g., Pokémon). | ¶71 | col. 8:45-50 |
| an AR server coupled with the AR object repository... is configured to: | The platform includes a "SERVER RUNTIME" and "AR CLOUD" that are coupled with and access the "AR DATA" repository. | ¶71 | col. 21:51-52 |
| obtain digital data representative of an environment of an AR capable mobile device, the digital data including a device location of the AR capable device and a virtual element attribute | The apps display a "Map View" depicting the real-world location of the mobile device (represented by an avatar) and nearby AR objects (Pokémon). The specific type of Pokémon is alleged to be a virtual element attribute. | ¶72 | col. 4:1-14 |
| determine at least one context related to the AR capable device and pertinent to the environment based at least on the device location | The apps consider the real-world environment, such as the device's proximity to a body of water or the local weather, as a context that influences gameplay. A screenshot in the complaint shows a "Sunny" weather condition boosting certain Pokémon types (Compl. ¶76, p. 25). | ¶74 | col. 4:10-14 |
| identify relevant AR objects from the AR object repository representing available AR objects corresponding to the at least one context | Based on the determined context (e.g., sunny weather), the app identifies corresponding relevant AR objects (e.g., grass, ground, and fire-type Pokémon) to appear more frequently. | ¶76 | col. 4:14-19 |
| determine whether to alter presence of a relevant AR object based on at least the device location and the virtual element attribute | The app determines whether to display a Pokémon AR object based on its proximity to the user's avatar. The complaint provides a screenshot showing a Squirtle AR object displayed on the map because it is in proximity to the user (Compl. ¶78, p. 26). | ¶78 | col. 17:1-18 |
| cause the AR capable device to render the relevant AR object according to its altered presence | The app causes the mobile device to display or not display the Pokémon AR objects on the Map View based on the prior determination of altered presence. | ¶80 | col. 22:1-3 |
- Identified Points of Contention:
- Scope Questions: A central question may be whether the term "interference," which the patent describes with analogies to "constructive" and "destructive" effects, can be construed to read on the accused products' use of contextual rules (e.g., if weather is "sunny," increase spawn rate of "fire-type" Pokémon). The dispute may focus on whether this game logic constitutes the "ambient interplay among present, or relevant, elements in a scene" described by the patent (’051 Patent, col. 3:29-32).
- Technical Questions: The analysis may question whether the alleged "virtual element attribute" (e.g., a Pokémon's type) corresponds to the technical meaning of that term as used and defined within the patent specification.
’518 Patent Infringement Allegations
| Claim Element (from Independent Claim 1) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| A device capable of rendering augmented reality (AR)... comprising: at least one sensor, including a location sensor; a display; a non-transitory computer readable memory... and at least one processor... | The Accused Products operate on mobile devices (e.g., phones, tablets) that contain these hardware components, in conjunction with Niantic's backend servers. | ¶95 | col. 25:50-58 |
| obtain sensor data from the at least one sensor wherein the sensor data includes a device location obtained from the location sensor | The apps obtain location information from the mobile device's location sensors (e.g., GPS). | ¶96 | col. 26:1-3 |
| access an area tile map of the area of interest, the area tile map represented by a set of tile subareas that includes one or more tessellated tiles from a tessellated tile map | The apps allegedly access OpenStreetMaps tile data, which is described as having tile subareas arranged in a "tessellation." | ¶99 | col. 26:5-8 |
| identify a tile subarea from the set of tile subareas based at least in part on the device location relative to one or more locations of tile subareas | The app identifies a tile subarea corresponding to an in-game location, such as a Pokéstop, based on the device's location. | ¶100 | col. 26:9-12 |
| wherein one or more tessellated tiles within the identified tile subarea are associated with one or more AR content objects | The tessellated tiles corresponding to the location of the identified Pokéstop are alleged to be associated with that Pokéstop, which is an AR content object. | ¶101 | col. 26:14-16 |
| populate the non-transitory computer readable memory with at least one of the one or more AR content objects associated with the one or more tessellated tiles | The app interacts with the Niantic backend platform to load data for AR content objects (e.g., a Pokémon's AR image) into the mobile device's memory. | ¶102 | col. 26:16-20 |
| render the at least one of the one or more AR content objects that is associated with the identified tile subarea on the display based on a view of interest | The app depicts AR content objects, such as a Pokéstop or Gym, on the mobile device's display. | ¶103 | col. 26:20-23 |
- Identified Points of Contention:
- Scope Questions: A potential point of contention is whether a generic, grid-based mapping service like OpenStreetMaps qualifies as the claimed "area tile map" that includes "tessellated tiles from a tessellated tile map." The defense may argue that the patent requires a specialized AR-centric map structure based on "experience clusters," a concept described in the specification (’518 Patent, col. 2:6-8), rather than a standard geographic grid.
- Technical Questions: The infringement theory identifies an in-game point of interest (a "Pokéstop") as a "tile subarea." The court may need to determine if a feature located within a map tile is technically synonymous with the "tile subarea" itself as claimed in the patent.
V. Key Claim Terms for Construction
For the ’051 Patent:
- The Term: "interference"
- Context and Importance: This term is central to the novelty of the invention as described in the patent. The viability of the infringement claim depends on construing this term to cover the accused games' use of contextual game logic to determine AR object appearances. Practitioners may focus on this term because it is not a standard term of art in software and its definition will likely dictate the outcome of the infringement analysis.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The specification provides an abstract definition: "Interference represents ambient interplay among present, or relevant, elements in a scene that gives rise to an augmented reality experience" (’051 Patent, col. 3:29-32). This language could be argued to encompass any system where environmental factors influence virtual object presentation.
- Evidence for a Narrower Interpretation: The patent repeatedly uses analogies to physics, such as "constructive interference" leading to "enhanced presence" and "destructive interference" leading to "suppressed presence" (’051 Patent, Abstract, Fig. 4). This could support a narrower definition requiring a system that more closely mimics wave interference principles, rather than simple conditional logic.
For the ’518 Patent:
- The Term: "tessellated tile map"
- Context and Importance: The infringement allegation relies on equating a standard third-party mapping service (OpenStreetMaps) with this claim term. The dispute will likely turn on whether the patent claims any grid-based map or one with specific, AR-related structural properties.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The claim language recites an "area tile map... that includes one or more tessellated tiles from a tessellated tile map," which could plausibly describe any map composed of a grid of tiles (’518 Patent, col. 26:5-8). The complaint's definition of "tessellated" as "composed of a pattern of repeated shapes that fit together closely without gaps or overlaps" supports this reading (Compl. ¶39).
- Evidence for a Narrower Interpretation: The patent’s summary of the invention states that known systems "fail to contemplate segmenting an area into clusters based on what is viewable or what AR content is available" and that a tile map can be "tessellated based on the experience clusters" (’518 Patent, col. 2:5-8, col. 2:37-38). This suggests the tessellation itself may need to be derived from AR content density or "experience clusters," not just a generic geographical grid.
VI. Other Allegations
- Indirect Infringement: The complaint alleges that Niantic induces infringement by providing the Accused Products to end-users and encouraging their use through instructions, promotional materials, and technical support websites (Compl. ¶¶59, 62).
- Willful Infringement: Willfulness allegations are based on Niantic's alleged continued infringement after receiving actual notice of the Asserted Patents, at the latest upon service of the initial complaint on or about September 2020 (Compl. ¶¶60, 67, 87). The complaint references "September 38, 2020," which appears to be a typographical error (Compl. ¶60).
VII. Analyst’s Conclusion: Key Questions for the Case
- A core issue will be one of definitional scope: can the patent term "interference," described in the ’051 Patent with analogies to constructive and destructive wave physics, be construed to cover the contextual rule-based system (e.g., weather, location) used by the accused games to determine the appearance of virtual characters?
- A second key issue will be one of technical implementation: does a standard, grid-based mapping system like OpenStreetMaps meet the "tessellated tile map" limitation of the ’518 Patent, or does the patent’s specification require a more specialized map structure derived from AR-specific "experience clusters"?
- An evidentiary question will be one of functional correspondence: do the points of interest in the accused games (e.g., "Pokéstops") function as the claimed "tile subareas" of the ’518 Patent, or are they merely AR objects located within a broader tile subarea, creating a mismatch with the claim language?