DCT

1:24-cv-02026

Electronic Scripting Products Inc v. Aw Faber Castell USA Inc

Log In

I. Executive Summary and Procedural Information

  • Parties & Counsel:
  • Case Identification: 1:24-cv-02026, N.D. Ohio, 11/20/2024
  • Venue Allegations: Venue is based on allegations that the Defendant maintains a regular and established place of business in the district and has committed acts of infringement there.
  • Core Dispute: Plaintiff alleges that Defendant’s augmented reality coloring applications infringe patents related to determining the absolute position and orientation (pose) of a manipulated device, such as a smartphone, within a real-world three-dimensional environment.
  • Technical Context: The technology enables a handheld device's camera and sensors to track its pose relative to physical objects, a foundational capability for the augmented reality market that overlays digital information onto a user's view of the real world.
  • Key Procedural History: The complaint does not mention any prior litigation, inter partes review proceedings, or licensing history related to the patents-in-suit.

Case Timeline

Date Event
2004-01-30 U.S. Patent No. 7,826,641 Earliest Priority Date
2004-01-30 U.S. Patent No. 10,191,559 Earliest Priority Date
2010-11-02 U.S. Patent No. 7,826,641 Issues
2016-01-01 "Never Ending Forest" App Launch Year Mentioned in Case Study
2019-01-29 U.S. Patent No. 10,191,559 Issues
2023-10-01 "ARt Alive" App Update Date
2024-11-20 Complaint Filed

II. Technology and Patent(s)-in-Suit Analysis

U.S. Patent No. 10,191,559 - "Computer Interface For Manipulated Objects With An Absolute Pose Detection Component"

  • Patent Identification: U.S. Patent No. 10191559, “Computer Interface For Manipulated Objects With An Absolute Pose Detection Component,” issued January 29, 2019. (Compl. ¶7).

The Invention Explained

  • Problem Addressed: The patent’s background section describes the need for an “efficient, accurate and low-cost” method of determining the absolute pose (position and orientation) of handheld objects used to interface with the digital world, noting that prior art solutions were often computationally expensive or unable to track all six degrees of freedom. (’559 Patent, col. 2:21-34).
  • The Patented Solution: The invention proposes using a manipulated object, such as a smartphone, equipped with an on-board photodetector (camera) to detect high optical contrast features in the environment. A controller analyzes the "derivative pattern" of these features—how their appearance changes from the camera's perspective—to calculate the object's position and orientation. This optical data can be supplemented by information from other sensors, such as an inertial measurement unit. (’559 Patent, Abstract; col. 6:28-40).
  • Technical Importance: This approach provided a method for robust six-degree-of-freedom tracking using the standard components found in consumer mobile devices, which was a critical enabler for mass-market augmented reality applications. (’559 Patent, col. 2:21-34).

Key Claims at a Glance

  • The complaint asserts independent claim 1. (Compl. ¶9).
  • Essential elements of Claim 1 include:
    • A manipulated object cooperating with a first plurality of high optical contrast features, comprising:
      • a) a photodetector configured to detect the features and generate data representative of their positions;
      • b) a controller configured to identify a derivative pattern of the features from the photodetector data, with the derivative pattern being indicative of the photodetector's position; and
      • c) at least one component selected from an auxiliary motion detection component, an active illumination component, or a scanning component.
  • The complaint also asserts dependent claims 6, 7, 10, 15, 16, 19, 24, and 25. (Compl. ¶21).

U.S. Patent No. 7,826,641 - "Apparatus And Method For Determining An Absolute Pose Of A Manipulated Object In A Real Three-Dimensional Environment With Invariant Features"

  • Patent Identification: U.S. Patent No. 7826641, “Apparatus And Method For Determining An Absolute Pose Of A Manipulated Object In A Real Three-Dimensional Environment With Invariant Features,” issued November 2, 2010. (Compl. ¶13).

The Invention Explained

  • Problem Addressed: The patent identifies a major problem with prior art manipulated objects: the lack of a robust and rapid system for "one-to-one motion mapping between real space and cyberspace," which requires capturing all six degrees of freedom of the object's motion. (’641 Patent, col. 1:45-64).
  • The Patented Solution: The invention describes an apparatus that includes an on-board "optical measuring means," such as a camera, on the manipulated object. This component optically infers the object's absolute pose by observing at least one "invariant feature" (a fixed, recognizable point or pattern) in the environment. The determined pose, expressed as Euler angles and world coordinates, is then prepared by a processor and sent to an application via a communication link, allowing the object's physical movements to control the application. (’641 Patent, Abstract; col. 9:1-14).
  • Technical Importance: The patent discloses a foundational system for using an on-board optical sensor to derive the full six-degree-of-freedom pose of an object relative to static environmental features, enabling the direct mapping of real-world object manipulation to a digital context. (’641 Patent, col. 2:45-56).

Key Claims at a Glance

  • The complaint asserts independent claim 1. (Compl. ¶15).
  • Essential elements of Claim 1 include:
    • An apparatus for processing absolute pose data from a manipulated object, comprising:
      • a) at least one invariant feature in the environment;
      • b) an on-board optical measuring means for optically inferring the absolute pose using the invariant feature and expressing it with absolute pose data;
      • c) a processor for preparing the absolute pose data and identifying a subset thereof; and
      • d) a communication link for transmitting the subset to an application.
  • The complaint also asserts dependent claim 29. (Compl. ¶34).

III. The Accused Instrumentality

Product Identification

  • The accused products are the augmented reality applications “Colour to Life,” “Never Ending Forest,” and “ARt Alive.” (Compl. ¶10).

Functionality and Market Context

  • The accused applications function by using the camera of a mobile device (e.g., an iOS or Android device) to view physical coloring pages or templates. (Compl. ¶11). The applications recognize markings, such as QR codes, on the pages and overlay interactive 3D animations onto the live camera feed, creating the illusion that a user's colored drawing has come to life. (Compl. p. 4). To maintain the alignment of the digital overlay, the applications use the device's camera, processor, and internal motion sensors (such as an IMU) to continuously track the device's position and orientation relative to the physical page. (Compl. ¶¶ 11, 17). The complaint notes that the "Never Ending Forest" app achieved 600,000 downloads in 90 days, suggesting market adoption. (Compl. p. 6). The screenshot from the "Colour to Life" marketing materials illustrates the app in use, showing a colored character on a page and its animated 3D counterpart on the device screen. (Compl. p. 4).

IV. Analysis of Infringement Allegations

U.S. Patent No. 10,191,559 Infringement Allegations

Claim Element (from Independent Claim 1) Alleged Infringing Functionality Complaint Citation Patent Citation
A manipulated object cooperating with a first plurality of high optical contrast features... An iOS or Android mobile device used with an augmented reality set, such as the Colour to Life coloring book. ¶11 Abstract
a) a photodetector configured to detect said first plurality of high optical contrast features and generate photodetector data... The camera of the mobile device, which detects high contrast features and markings like QR codes on the coloring page. The complaint includes a marketing image showing the QR codes on the "Colour to Life" packaging. (Compl. p. 4). ¶11 col. 6:28-32
b) a controller configured to identify a derivative pattern of said first plurality of high optical contrast features... wherein said derivative pattern is indicative of the position of said photodetector... The processing unit(s) of the iOS or Android device, which allegedly identifies a derivative pattern from the camera data to determine the position of the camera. ¶11 col. 6:32-34
c) at least one component selected from the group consisting of an auxiliary motion detection component... The mobile device's auxiliary motion detection components, such as its Inertial Measurement Unit (IMU) or motion sensing unit. ¶11 Abstract

U.S. Patent No. 7,826,641 Infringement Allegations

Claim Element (from Independent Claim 1) Alleged Infringing Functionality Complaint Citation Patent Citation
An apparatus for processing absolute pose data derived from an absolute pose of a manipulated object... A mobile device, such as an iOS or Android device, held by a user. ¶17 Abstract
a) at least one invariant feature in said real three-dimensional environment; Markings, including QR codes, on the augmented reality coloring set. ¶17 col. 9:35-39
b) an optical measuring means for optically inferring said absolute pose from on-board said manipulated object using said at least one invariant feature and expressing said inferred absolute pose with absolute pose data... The camera of the iOS or Android device, which is alleged to infer the device's orientation (pitch, yaw, roll) and position (x,y,z) relative to a reference location. The complaint offers a case study image for the "Never Ending Forest" app showing the on-screen result of this tracking. (Compl. p. 12). ¶17 col. 9:15-22
c) a processor for preparing said absolute pose data and identifying a subset of said absolute pose data; The processing unit(s) of the mobile device, which allegedly prepares the absolute pose data. ¶17 col. 10:25-28
d) a communication link for transmitting said subset to an application. An internal communication link within the mobile device that transmits the pose data to the accused augmented reality application. The complaint includes app store screenshots for "ARt Alive" showing the application that receives the data. (Compl. p. 13). ¶17 col. 10:31-35

Identified Points of Contention

  • Scope Questions: For the ’559 Patent, a central issue may be the construction of "derivative pattern." The dispute may focus on whether this term requires a specific type of geometric analysis of a known pattern's perspective distortion, or if it can be read more broadly to cover other computer vision techniques for determining position from camera data.
  • Technical Questions: For the ’641 Patent, a key technical question will be whether the accused products internally compute and express pose using the specific format required by the claim: "absolute pose data (φ, θ, ψ, x, y, z) representing Euler rotated object coordinates expressed in world coordinates." The complaint alleges this occurs but provides evidence based on the external functionality of the apps rather than their internal data processing methods.

V. Key Claim Terms for Construction

  • The Term: "derivative pattern" (’559 Patent, Claim 1)

  • Context and Importance: This term is central to how the "controller" determines the "position" of the photodetector. Its construction will likely define the scope of infringement for claim 1(b). Practitioners may focus on this term because its definition could distinguish between specific mathematical transformations of a known visual pattern versus more general feature-matching or machine-learning-based tracking algorithms.

  • Intrinsic Evidence for Interpretation:

    • Evidence for a Broader Interpretation: The specification describes the derivative pattern as being "indicative of the asymmetric and generally linear pattern" of light sources, suggesting it relates to the transformed appearance of a known configuration. (’559 Patent, col. 6:32-35).
    • Evidence for a Narrower Interpretation: The specification states that as the photodetector's pose changes, the source pattern "undergoes a well-understood transformation (i.e., perspective distortion plus any optical aberrations...)." (’559 Patent, col. 6:35-39). A party could argue that "derivative pattern" is limited to the result of analyzing this specific type of geometric transformation, not just any data that correlates with position.

  • The Term: "optical measuring means for optically inferring said absolute pose" (’641 Patent, Claim 1)

  • Context and Importance: This limitation is drafted in means-plus-function format under 35 U.S.C. § 112, ¶ 6 (pre-AIA). Its scope is therefore limited to the corresponding structures disclosed in the specification and their equivalents. The infringement analysis will depend on whether the accused structure (a smartphone camera and processor) is equivalent to the structures disclosed in the patent.

  • Intrinsic Evidence for Interpretation:

    • Evidence for a Broader Interpretation: The specification describes the structure in general terms, such as an "on-board optical measuring arrangement" with a "light-measuring component with a lens and an optical sensor that form an imaging system." (’641 Patent, col. 9:15-16, col. 10:17-19). This high-level description could be argued to cover a standard smartphone camera.
    • Evidence for a Narrower Interpretation: The scope of a means-plus-function claim is limited to the specific embodiments disclosed. A party may argue the term is limited to the detailed structures described in the specification and its parent applications, potentially including specific sensor types (like a position-sensitive device or PSD) or algorithms, and that the accused products use non-equivalent structures.

VI. Other Allegations

  • Indirect Infringement: The complaint alleges induced infringement for both the ’559 and ’641 patents. The allegations are based on Faber-Castell providing the accused applications along with "specific instructions or training" that allegedly cause end-users to directly infringe the patents by operating the apps as intended. (Compl. ¶¶ 26-29, 39-42).
  • Willful Infringement: Willfulness allegations for both patents are based on alleged knowledge "since at least the date of the filing of this Complaint." (Compl. ¶22, ¶35). This pleading supports a claim for post-suit willful infringement.

VII. Analyst’s Conclusion: Key Questions for the Case

  • A core issue will be one of claim scope: can the term "derivative pattern" in the ’559 Patent, which is described in the context of analyzing geometric transformations like perspective distortion, be construed to cover the potentially different computer vision and tracking algorithms used in modern augmented reality applications?
  • A key evidentiary question will be one of technical proof: what evidence can be produced to show that the accused applications, internally, perform the specific data processing and formatting steps required by the claims—such as identifying a "derivative pattern" (’559 Patent) or expressing pose data using "Euler rotated object coordinates" (’641 Patent)—beyond merely achieving a similar functional result?