DCT

3:23-cv-04070

Hawkvision eMMERSION Computing LLC v. Tangible Play Inc

Log In

I. Executive Summary and Procedural Information

  • Parties & Counsel:
  • Case Identification: 5:23-cv-04070, N.D. Cal., 08/10/2023
  • Venue Allegations: Plaintiff alleges venue is proper in the Northern District of California because Defendant has its principal place of business in the district, regularly conducts business there, and has committed the alleged acts of infringement within the district.
  • Core Dispute: Plaintiff alleges that Defendant’s Osmo line of educational toys infringes a patent related to systems that capture real-world user interactions on a physical surface and combine them with computer-generated images on a display.
  • Technical Context: The technology operates in the field of augmented reality and tangible user interfaces, which seek to create more intuitive human-computer interactions by merging physical and digital worlds, offering an alternative to traditional input devices like the mouse and keyboard.
  • Key Procedural History: The complaint notes that Defendant Tangible Play has asserted its own later-filed patents are practiced by the accused Osmo products, which Plaintiff contends supports the validity of the asserted patent. The complaint also states that Plaintiff provided Defendant with notice of the patent-in-suit via a letter dated May 11, 2023, approximately three months prior to filing the lawsuit.

Case Timeline

Date Event
2009-05-22 '391 Patent Priority Date
2012-10-01 Defendant's Alleged Patent Priority Date (approximate)
2014-06-24 '391 Patent Issue Date
2023-05-11 Plaintiff's Notice Letter to Defendant
2023-08-10 Complaint Filing Date

II. Technology and Patent(s)-in-Suit Analysis

U.S. Patent No. 8,760,391 - "Input Cueing Emersion System and Method"

  • Patent Identification: U.S. Patent No. 8,760,391, "Input Cueing Emersion System and Method," issued June 24, 2014.

The Invention Explained

  • Problem Addressed: The patent describes the limitations of conventional computer input devices, such as the mouse and keyboard, which can be difficult to master, counter-intuitive for tasks like drawing, and create a disconnect between a user's physical actions and the on-screen result ('391 Patent, col. 1:43-52; Compl. ¶3). The invention aims to solve the problem of creating a more natural and ergonomic way to interact with a computer, particularly for artistic creation (Compl. ¶¶2-3).
  • The Patented Solution: The invention proposes a system that uses an image capturing device (like a camera) in combination with mirrors or reflectors to view a user's hands and tools on a physical input surface in front of a display ('391 Patent, Abstract). The system captures a live image of this physical interaction, combines it with computer-generated graphics (like a virtual keyboard or drawing), and shows the merged, "emmersed" view to the user in real-time on the display, making it appear as if their hands are interacting directly with the digital content ('391 Patent, col. 2:2-7; Compl. ¶3).
  • Technical Importance: This approach seeks to provide a more intuitive and efficient user experience by allowing a user to see their hands and their work simultaneously in the same visual plane, eliminating the need to look back and forth between a physical input device and a separate screen (Compl. ¶¶3, 5).

Key Claims at a Glance

  • The complaint asserts independent claim 31 (Compl. ¶14).
  • The essential elements of independent claim 31 include:
    • An electronic controller, a processing component, and a display.
    • An input device with an input area.
    • A "primary imaging system" (e.g., a mirror) positioned over the input area.
    • A "secondary imaging system" (e.g., a camera) that receives a reflected image from the primary system.
    • A "combiner" that receives a first image signal (from the imaging systems) and a second image signal (from the electronic controller) and merges them into a combined, synchronized signal.
    • A software program that produces the second image and displays the combined image.
  • The complaint notes that its infringement theory is an "example and is not limiting," reserving the right to assert other claims (Compl. ¶16).

III. The Accused Instrumentality

Product Identification

  • The "Osmo line of products," which includes a "reflector," a "base," physical game pieces, and software applications. When these components are used with an Apple iPad or Amazon Fire tablet, they form the "Accused Osmo Device" (Compl. ¶5).

Functionality and Market Context

  • The Osmo system uses a reflector that clips over the tablet's front-facing camera, redirecting its view to the physical surface in front of the tablet (Compl. ¶14[d]-[e]). The Osmo software applications process the video feed of the user interacting with physical objects (e.g., drawing on a board, arranging letter tiles) in this physical space. This physical interaction is then integrated into an on-screen digital game or learning activity (Compl. ¶5). The complaint includes a photograph showing a user drawing on a physical surface, with the drawing appearing inside a digital environment on the tablet's screen (Compl. p. 8). The complaint alleges significant commercial success, noting Tangible Play's acquisition for $120 million and widespread distribution through major retailers (Compl. ¶¶4, 16(b)).

IV. Analysis of Infringement Allegations

'391 Patent Infringement Allegations

| Claim Element (from Independent Claim 31) | Alleged Infringing Functionality - | Complaint Citation | Patent Citation |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |:-----------------------|:-------------------|
| an electronic controller - | The iPad and Fire tablets, with Osmo applications installed, which control the system components. - | ¶14[a] | col. 1:13-17 |
| a processing component connected to the electronic controller - | The CPUs within the iPad and Fire tablets (e.g., A13, A14, M1, M2 processors). - | ¶14[b] | col. 16:32-33 |
| a display connected to the electronic controller - | The screens of the iPad and Fire tablets. - | ¶14[c] | col. 16:40-41 |
| an input device with an input area... connected to the electronic controller - | The combination of the tablet's front-facing camera and the Osmo reflector, with the "input area" being the physical working space in front of the tablet. The complaint provides a photo illustrating this input area (Compl. p. 6). | ¶14[d] | col. 11:7-10 |
| a primary imaging system disposed substantially over the input area and aimed at the input area - | The Osmo reflector and its mirror, which is positioned over the input area. The complaint includes a visual depicting the reflector placed over the tablet camera (Compl. p. 7). | ¶14[e] | col. 12:59-62 |
| a secondary imaging system disposed within a receiving image field of the primary imaging system, wherein the secondary imaging system receives the first image from the primary imaging system, captures the first image and transmits the first image... to the combiner | The front-facing camera of the iPad or Fire tablet, which is positioned to receive the reflected image from the Osmo reflector, capture it, and transmit it to the software. - | ¶14[f] | col. 13:1-13 |
| a combiner for receiving a first image... and a second image... and combining the first and second image signals to create a combined, synchronized signal - | The Osmo application software, which allegedly receives the camera image signal and combines it with a second signal representing application-generated graphics and animation. - | ¶14[g] | col. 10:30-43 |
| an X, Y and/or Z input display program or operating system... wherein the input data program receives the post processed X, Y and/or Z data from the input device, and produces the second image; wherein the combined signal is received by the display and a combined image is depicted on the display | The Osmo applications, which are loaded into the tablet's memory, process the camera data, produce the computer-generated graphics, and send the final combined image to the tablet display. An image in the complaint depicts this combined output on the screen (Compl. p. 8). | ¶14[h] | col. 10:44-52 |

  • Identified Points of Contention:
    • Scope Questions: The patent frequently discusses a "drawing tablet" or "drawing surface" as the input medium ('391 Patent, col. 11:7-10). A question for the court may be whether the claim term "input device with an input area" can be construed to cover the accused system, where the input area is a generic physical surface like a tabletop rather than a dedicated electronic tablet.
    • Technical Questions: Claim 31 recites a "primary imaging system" that provides a "first image output signal" to a "combiner". The complaint alleges the passive Osmo reflector is this system. This raises the question of what evidence supports the allegation that a passive mirror generates an "output signal" in the manner required by the claim, as opposed to merely redirecting light to the "secondary imaging system" (the camera), which is the component that generates the electronic signal.

V. Key Claim Terms for Construction

  • The Term: "primary imaging system"

  • Context and Importance: The infringement reading hinges on whether the accused product's passive mirror/reflector meets this definition. The complaint alleges it does (Compl. ¶14[e]). However, the subsequent "combiner" limitation requires this system to provide an "image output signal," which may raise a dispute over whether a passive component can satisfy this functional requirement.

  • Intrinsic Evidence for Interpretation:

    • Evidence for a Broader Interpretation: The specification broadly defines "imaging" to include "prisms... mirrors, reflectors, whether in any curved manner or other optical type" ('391 Patent, col. 5:10-16), language which may support including a passive reflector within the term's scope.
    • Evidence for a Narrower Interpretation: The claim's requirement that the "combiner" receives a "first image output signal from the primary imaging system" ('391 Patent, col. 23:53-55) may suggest the system must be an active component capable of generating an electronic signal, not just redirecting photons. A party could argue this limits the term to a device like a camera.

  • The Term: "combiner"

  • Context and Importance: This term is central to how the patented invention merges the real and virtual worlds. The complaint identifies the "Osmo application software" as the combiner (Compl. ¶14[g]). Practitioners may focus on this term because its definition dictates the specific inputs required, and a mismatch could present a non-infringement argument.

  • Intrinsic Evidence for Interpretation:

    • Evidence for a Broader Interpretation: The patent abstract describes a "combiner module" in general terms, used to "combine the visual action... with the visual output from a computer" ('391 Patent, Abstract), which could support a flexible, software-based interpretation.
    • Evidence for a Narrower Interpretation: Claim 31 requires the combiner to receive two distinct inputs: "a first image output signal from the primary imaging system" and "a second image output signal from the electronic controller" ('391 Patent, col. 23:52-58). The architectural diagram in Figure 54 shows distinct signal paths into the "VIDEO COMBINER," which could be used to argue for a more rigid two-input structure that a defendant's software may not possess.

VI. Other Allegations

  • Indirect Infringement: The complaint alleges inducement by asserting that Defendant provides instructional materials, marketing, and videos that actively encourage and guide end-users to assemble and use the Osmo components with a tablet in an infringing manner (Compl. ¶16(b)). It also alleges contributory infringement, claiming the Osmo reflector and applications are a material part of the invention, are not staple articles of commerce, and have no substantial non-infringing use (Compl. ¶17(b)).
  • Willful Infringement: Willfulness is alleged based on both pre-suit and post-suit knowledge. The complaint alleges pre-suit knowledge from at least May 11, 2023, when Plaintiff sent a notice letter (Compl. ¶16(a)). It further alleges Defendant continued its infringing activities after receiving notice and after the filing of the lawsuit, without attempting to design around the patent (Compl. ¶18).

VII. Analyst’s Conclusion: Key Questions for the Case

  • A core issue will be one of definitional scope: can the term "primary imaging system", which is functionally required by the claim to provide an "image output signal," be construed to read on the accused product's passive, non-electronic mirror?
  • A key evidentiary question will be one of architectural equivalence: does the accused Osmo software embody the specific two-input "combiner" architecture recited in Claim 31, or does its method for merging physical and digital information differ in a way that falls outside the literal scope of the claim?
  • The outcome may also depend on claim construction: the court's interpretation of whether the patent's language, particularly terms like "imaging system" and "combiner", requires a specific hardware and signal-flow configuration or allows for a more flexible, functional interpretation that could encompass the accused software-centric system.