DCT
2:24-cv-01082
Electronic Scripting Products Inc v. Fresh Consulting Inc
I. Executive Summary and Procedural Information
- Parties & Counsel:
- Plaintiff: Electronic Scripting Products, Inc. (Delaware)
- Defendant: Fresh Consulting, Inc. (Washington)
- Plaintiff’s Counsel: Banie & Ishimoto LLP
- Case Identification: 2:24-cv-01082, W.D. Wash., 07/23/2024
- Venue Allegations: Plaintiff alleges venue is proper because Defendant maintains a regular and established place of business within the Western District of Washington.
- Core Dispute: Plaintiff alleges that Defendant’s augmented and virtual reality applications and associated software infringe three patents related to determining the absolute position and orientation of manipulated objects for computer interaction.
- Technical Context: The technology at issue involves using on-board optical sensors, such as cameras in smartphones or headsets, to track features in the user's environment to determine the device's six-degree-of-freedom pose for augmented and virtual reality applications.
- Key Procedural History: The complaint does not mention any prior litigation, Inter Partes Review (IPR) proceedings, or licensing history related to the patents-in-suit.
Case Timeline
| Date | Event |
|---|---|
| 2004-01-30 | Earliest Priority Date for '641 & '540 Patents |
| 2006-03-08 | Earliest Priority Date for '559 Patent |
| 2010-11-02 | U.S. Patent No. 7,826,641 Issued |
| 2016-01-05 | U.S. Patent No. 9,229,540 Issued |
| 2019-01-29 | U.S. Patent No. 10,191,559 Issued |
| 2024-07-23 | Complaint Filed |
II. Technology and Patent(s)-in-Suit Analysis
U.S. Patent No. 10,191,559 - "Computer Interface For Manipulated Objects With An Absolute Pose Detection Component"
- Patent Identification: U.S. Patent No. 10,191,559, titled “Computer Interface For Manipulated Objects With An Absolute Pose Detection Component,” issued on January 29, 2019 (the ’559 Patent).
The Invention Explained
- Problem Addressed: The patent addresses the need for a low-cost, robust, and accurate apparatus for determining the absolute position and orientation (pose) of hand-held objects, such as pointers or tablets, used for interfacing with a digital world, noting that prior systems were often computationally expensive or lacked true six-degree-of-freedom tracking (Compl. ¶7; ’559 Patent, col. 1:26-31, col. 2:59-63).
- The Patented Solution: The invention proposes a manipulated object with an on-board photodetector (e.g., a camera) that detects "high optical contrast features" in the surrounding environment. An on-board controller analyzes the detected features, identifies a "derivative pattern" from the sensor data that changes as the object moves, and uses this pattern to determine the object's absolute pose. The system can be supplemented by other sensors, such as an inertial measurement unit, for improved performance (’559 Patent, Abstract; col. 5:26-43).
- Technical Importance: This approach aimed to enable absolute pose tracking on consumer devices by relying on a single on-board optical sensor and environmental features, a foundational concept for modern "inside-out" tracking in mobile augmented reality (AR) (’559 Patent, col. 2:4-14).
Key Claims at a Glance
- The complaint asserts independent Claim 1 (Compl. ¶9).
- The essential elements of Claim 1 are:
- A manipulated object cooperating with a first plurality of high optical contrast features disposed in a real three-dimensional environment.
- a) a photodetector configured to detect said features and generate data representative of their positions.
- b) a controller configured to identify a derivative pattern from the photodetector data, where the derivative pattern indicates the photodetector's position.
- c) at least one component from the group of an auxiliary motion detection component, an active illumination component, or a scanning component.
- The complaint also asserts dependent claims 6, 7, 10, 15, 16, 19, 24, and 25 (Compl. ¶¶ 11, 26).
U.S. Patent No. 7,826,641 - "Apparatus And Method For Determining An Absolute Pose Of A Manipulated Object In A Real Three-Dimensional Environment With Invariant Features"
- Patent Identification: U.S. Patent No. 7,826,641, titled “Apparatus And Method For Determining An Absolute Pose Of A Manipulated Object In A Real Three-Dimensional Environment With Invariant Features,” issued on November 2, 2010 (the ’641 Patent).
The Invention Explained
- Problem Addressed: The patent identifies a major problem with prior art manipulated objects (like gaming wands) was their lack of a "sufficiently robust and rapid absolute pose determination system," often relying instead on relative motion sensors which could not support one-to-one mapping between real space and cyberspace (’641 Patent, col. 1:42-51, col. 2:5-9).
- The Patented Solution: The invention describes an apparatus on a manipulated object that uses an "optical measuring means" to optically infer its absolute pose (including position and Euler rotation angles) by observing at least one "invariant feature" in the environment. A processor prepares this pose data, identifies a required subset, and transmits it via a communication link to an application (’641 Patent, Abstract; col. 9:11-28).
- Technical Importance: The patent describes a self-contained system for determining a six-degree-of-freedom pose using environmental features, providing a framework for devices to orient themselves in space without external tracking infrastructure (’641 Patent, col. 1:26-34).
Key Claims at a Glance
- The complaint asserts independent Claim 1 (Compl. ¶14).
- The essential elements of Claim 1 are:
- An apparatus for processing absolute pose data derived from an absolute pose of a manipulated object.
- a) at least one invariant feature in the real three-dimensional environment.
- b) an optical measuring means on the object for optically inferring the absolute pose from the invariant feature and expressing it with absolute pose data (including Euler rotated object coordinates).
- c) a processor for preparing the absolute pose data and identifying a subset of it.
- d) a communication link for transmitting the subset to an application.
- The complaint also asserts dependent claim 29 (Compl. ¶¶ 17, 39).
U.S. Patent No. 9,229,540 - "Deriving Input From Six Degrees Of Freedom Interfaces"
- Patent Identification: U.S. Patent No. 9,229,540, titled “Deriving Input From Six Degrees Of Freedom Interfaces,” issued on January 5, 2016 (the ’540 Patent) (Compl. ¶18).
- Technology Synopsis: The patent addresses an interface for deriving input from an item's absolute pose in a 3D environment. The claimed solution involves a unit on the item receiving "non-collinear optical inputs" from at least one stationary object to establish a stable frame. Processing electronics then employ a computer vision algorithm using "homography" to recover the item's absolute pose from these inputs and generate a signal for an application (’540 Patent, Abstract).
- Asserted Claims: The complaint asserts independent Claim 1 (Compl. ¶20).
- Accused Features: The complaint alleges that Defendant's AR/VR headsets (e.g., Oculus Quest, Microsoft HoloLens) infringe by using on-board cameras to receive optical inputs from the user's environment to establish a stable frame and determine the headset's absolute pose using computer vision algorithms (Compl. ¶¶ 21-22).
III. The Accused Instrumentality
Product Identification
- The complaint identifies the accused instrumentalities as Defendant’s "augmented reality apps and associated software and products" ("Accused Products") (Compl. ¶9). This category includes applications developed for iOS devices that use Apple's ARKit framework, Android devices that use Google's ARCore framework, and software for AR/VR headsets like the Oculus Quest and Microsoft HoloLens (Compl. ¶¶ 10, 13, 22).
Functionality and Market Context
- The Accused Products are alleged to utilize the hardware of mobile devices and headsets to provide AR and VR experiences (Compl. ¶10). Functionally, these products are alleged to use the device's camera as a photodetector to sense the real-world environment and its processing unit as a controller (Compl. ¶10). Frameworks like ARKit and ARCore are used to process the camera data, identify features in the environment, and determine the device's absolute pose (position and orientation) (Compl. ¶10). This pose information is then used by the application to overlay digital content on the real world or create an immersive virtual environment (Compl. ¶¶ 10, 16). The complaint provides a visual depicting a tablet running an AR application that overlays a digital model onto a real-world object (Compl. p. 6).
IV. Analysis of Infringement Allegations
’559 Patent Infringement Allegations
| Claim Element (from Independent Claim 1) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| A manipulated object cooperating with a first plurality of high optical contrast features disposed in a real three-dimensional environment, said manipulated object comprising: | The manipulated object is a mobile device like an iOS or Android phone, which cooperates with high contrast features in the environment, such as the drawings and markings on a book (Compl. p. 4) | ¶10 | col. 1:32-34 |
| a) a photodetector configured to detect said first plurality of high optical contrast features and generate photodetector data representative of the positions... | An iOS or Android device's camera acts as the photodetector, detecting high optical contrast features and generating data representing their positions. | ¶10 | col. 5:28-32 |
| b) a controller configured to identify a derivative pattern of said first plurality of high optical contrast features from said photodetector data... | The iOS or Android device's processing unit acts as the controller that identifies a derivative pattern from the camera data, which is indicative of the camera's position. | ¶10 | col. 5:32-37 |
| c) at least one component selected from the group consisting of an auxiliary motion detection component, an active illumination component and a scanning component. | The iOS or Android device's auxiliary motion detection components, such as an inertial measurement unit (IMU), satisfy this element. | ¶10 | col. 1:5-10 |
’641 Patent Infringement Allegations
| Claim Element (from Independent Claim 1) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| An apparatus for processing absolute pose data derived from an absolute pose of a manipulated object in a real three-dimensional environment, said apparatus comprising: | The apparatus is a mobile device (e.g., iPad/iPhone or Android) that processes pose data derived from its own pose. | ¶16 | col. 9:11-13 |
| a) at least one invariant feature in said real three-dimensional environment; | An "augmented reality book positioned in a real three-dimensional environment... with its drawings and markings" is alleged to be the invariant feature (Compl. p. 9). | ¶16 | col. 1:35-41 |
| b) an optical measuring means for optically inferring said absolute pose from on-board said manipulated object using said at least one invariant feature... | The device's camera serves as the optical measuring means to infer the device's absolute pose, including position (x,y,z) and orientation (pitch, yaw, roll). | ¶16 | col. 1:47-56 |
| c) a processor for preparing said absolute pose data and identifying a subset of said absolute pose data; and | The device's processing unit acts as the processor that prepares the absolute pose data and identifies a subset of it. | ¶16 | col. 9:14-16 |
| d) a communication link for transmitting said subset to an application. | An internal communication link within the mobile device transmits the subset of pose data to the running AR application. | ¶16 | col. 9:24-28 |
Identified Points of Contention
- Scope Questions: A central question for the '559 Patent may be whether the general environmental "features" detected by modern AR platforms like ARKit and ARCore meet the claim limitation of "high optical contrast features." For the ’641 Patent, a similar question arises as to whether algorithmically identified points on an object like an "augmented reality book" constitute an "invariant feature" as understood in the patent (Compl. ¶16).
- Technical Questions: For the ’559 Patent, a key technical question will be what evidence demonstrates that the accused ARKit/ARCore software identifies a "derivative pattern" from sensor data, as required by the claim, versus using other computational methods for pose estimation. For the ’641 Patent, the analysis may focus on whether the accused systems express pose using the specific "Euler rotated object coordinates" recited in the claim.
V. Key Claim Terms for Construction
The Term: "derivative pattern" (’559 Patent, Claim 1)
- Context and Importance: This term is central to the inventive step of the controller's function. The outcome of the infringement analysis for the ’559 Patent may depend on whether the complex feature tracking and world-mapping algorithms used by ARKit and ARCore can be characterized as identifying a "derivative pattern." Practitioners may focus on this term because the complaint's allegations are conclusory and do not explain how the accused software performs this specific step (Compl. ¶10).
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The specification states that the "derivative pattern is indicative of the asymmetric and generally linear pattern" and "enables one to correlate the asymmetric and generally linear pattern to the derivative pattern and obtain information about the pose of the photodetector" (’559 Patent, col. 5:35-43). This language could be argued to encompass any derived data set from the photodetector that allows for pose determination.
- Evidence for a Narrower Interpretation: The description of the derivative pattern is tied to a "well-understood transformation (i.e., perspective distortion plus any optical aberrations)" (’559 Patent, col. 5:37-41). This could support an argument that the term requires a specific type of geometric pattern analysis related to perspective changes, which may be narrower than the full scope of techniques used in modern AR systems.
The Term: "invariant feature" (’641 Patent, Claim 1)
- Context and Importance: The entire pose inference process of the ’641 Patent is anchored to this element. The complaint alleges that an "augmented reality book...with its drawings and markings" qualifies (Compl. ¶16). The viability of this infringement theory hinges on whether such an object meets the definition of an "invariant feature."
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The patent's background section broadly defines invariant features to include "stationary references such as ground planes, reference surfaces, lines, solids, fixed points and other invariant features" (’641 Patent, col. 1:35-39). This broad definition could support the plaintiff's position that any sufficiently stable and recognizable object in the environment qualifies.
- Evidence for a Narrower Interpretation: The detailed description provides more specific examples, such as "high optical contrast features such as edges of objects, special markings, or light sources" (’641 Patent, col. 9:28-31). A defendant may argue that this language implies the feature must be more distinct and structurally defined than the general drawings on a book.
VI. Other Allegations
- Indirect Infringement: The complaint alleges induced infringement for all three patents-in-suit. The basis for inducement is the allegation that the Defendant knowingly encourages and facilitates direct infringement by end-users when it markets, sells, and provides instructions for its AR/VR applications, which are designed to be used in an infringing manner (Compl. ¶¶ 31-34, 44-47, 57-60).
- Willful Infringement: Willfulness is alleged for all three patents. The complaint bases this allegation on the Defendant having "had knowledge and notice" of the patents and its own infringement "since at least the date of the filing of this Complaint" (Compl. ¶¶ 27, 29, 40, 42, 53, 55). This framing suggests a theory of post-suit willfulness rather than pre-suit knowledge.
VII. Analyst’s Conclusion: Key Questions for the Case
- A core issue will be one of technical specificity: will the evidence show that the general-purpose AR frameworks (ARKit/ARCore) used by the Accused Products perform the specific functions required by the patent claims, such as identifying a "derivative pattern" (’559 Patent) or employing "homography" (’540 Patent), or is there a fundamental mismatch in their technical operation?
- A second key issue will be one of definitional scope: can claim terms rooted in an earlier technological context, such as "high optical contrast features" (’559 Patent) and "invariant feature" (’641 Patent), be construed broadly enough to read on the algorithmically-generated environmental feature points utilized by modern, sophisticated AR and VR tracking systems?