DCT

1:24-cv-00589

RoboticVISIONTech Inc v. ABB Inc

Log In

I. Executive Summary and Procedural Information

  • Parties & Counsel:
  • Case Identification: 1:24-cv-00589, D. Del., 05/16/2024
  • Venue Allegations: Venue is alleged to be proper for ABB Inc. as it is a resident and corporate citizen of Delaware. For ABB Ltd., a Swiss corporation, venue is alleged to be proper in any judicial district as it is not a resident of the United States.
  • Core Dispute: Plaintiff alleges that Defendant’s robotic and discrete automation systems infringe three patents related to three-dimensional machine vision for robotic guidance.
  • Technical Context: The technology at issue involves using single or multiple cameras to determine an object's precise three-dimensional position and orientation to guide robotic arms in industrial automation settings.
  • Key Procedural History: The complaint alleges a long and contentious history between the parties, beginning with a 2006 licensing agreement between Defendant and Plaintiff's predecessor-in-interest. This was followed by failed license negotiations after Plaintiff acquired the patents, and a 2010 lawsuit filed by Defendant against Plaintiff concerning ownership of the underlying software, which Defendant later voluntarily dismissed. Plaintiff also notes a prior, ongoing patent infringement lawsuit it filed against Defendant in 2022 involving the same patents but focused on different accused products.

Case Timeline

Date Event
2002-05-24 Earliest Priority Date for ’755 and ’237 Patents
2004-07-14 Earliest Priority Date for ’814 Patent
2004-11-09 U.S. Patent No. 6,816,755 Issues
2006-05-01 Partnership agreement initiated between ABB and Plaintiff's predecessor
2008-02-26 U.S. Patent No. 7,336,814 Issues
2010-07-01 ABB files lawsuit against RVT in E.D. Michigan
2010-09-16 ABB voluntarily dismisses its lawsuit against RVT
2012-01-10 U.S. Patent No. 8,095,237 Issues
2013-01-01 ABB allegedly launches its Integrated Vision System
2015-01-01 ABB allegedly launches its YuMi products
2020-01-01 ABB allegedly launches its 3DQI products
2021-01-01 ABB allegedly launches its FlexLoader M family
2022-09-22 Plaintiff RVT files prior infringement suit against ABB Inc.
2024-05-16 Current Complaint Filing Date

II. Technology and Patent(s)-in-Suit Analysis

U.S. Patent No. 6,816,755 - "Method and Apparatus for Single Camera 3D Vision Guided Robotics," Issued November 9, 2004

The Invention Explained

  • Problem Addressed: The patent addresses the complexity, cost, and fragility of prior art 3D robotic vision systems, which often required multiple cameras (stereo vision) or specialized laser triangulation sensors that were ill-suited for harsh industrial environments (ʼ755 Patent, col. 1:12-33).
  • The Patented Solution: The invention proposes a method to determine an object's 3D pose using only a single, standard video camera mounted on a robot arm. The process involves three main stages: calibrating the camera to the robot's tool, "teaching" the system by selecting at least six visible features on a target object, and then using those taught features to find the object in 3D space and guide the robot's operational path (’755 Patent, Abstract; col. 2:45-65). The method relies on a framework of coordinate systems (e.g., "Camera Space," "Object Space," "Tool Frame") and calculating the mathematical transformations between them to orient the robot (’755 Patent, Fig. 2).
  • Technical Importance: This approach sought to make 3D vision guidance more accessible and robust for industrial manufacturing by reducing hardware costs and simplifying calibration compared to stereo-vision or laser-based systems (’755 Patent, col. 1:34-44).

Key Claims at a Glance

  • The complaint asserts independent claim 8 and dependent claims 1-7, 9-15, and 17-19 (Compl. ¶¶92-93).
  • Independent Claim 8 recites a method with three primary stages:
    • Calibrating the camera: This includes finding the camera's intrinsic parameters and its position relative to the robot's tool ("hand-eye" calibration).
    • Teaching object features: This involves capturing an image, selecting at least six visible features, calculating their 3D position to define an "Object Space," and computing transformations to relate the "Object Space" to the robot's "Tool Frame."
    • Object finding and positioning: This involves locating the taught features in a new image, computing the object's current location by transforming between the defined "spaces," and guiding the robot along an operational path.

U.S. Patent No. 7,336,814 - "Method and Apparatus for Machine-Vision," Issued February 26, 2008

The Invention Explained

  • Problem Addressed: The patent identifies a need for a 3D guidance system that eliminates the requirement for stereo cameras, inter-camera calibration, and a priori knowledge of the geometric relationships between all object features, which was often cumbersome and impractical (’814 Patent, col. 2:20-34).
  • The Patented Solution: The invention describes a method for 3D pose estimation that can use one or more cameras and works even with incomplete geometric data (i.e., "sparse model information"). The core idea is to determine the minimum number of additional camera views required to solve for the object's pose based on the number of features identified and the type of known physical relationships between them (e.g., distances, angles). A system of equations is then established and solved to find the 3D pose, providing flexibility when a complete object model is unavailable (’814 Patent, Abstract; col. 3:40-col. 4:11).
  • Technical Importance: The method allows for robust 3D pose estimation without requiring a complete CAD model or extensive manual measurements, making the system more adaptable to real-world manufacturing scenarios where such data may be incomplete or unavailable (’814 Patent, col. 5:50-64).

Key Claims at a Glance

  • The complaint asserts claims 1-39 (Compl. ¶102). Independent claim 1 is quoted in the complaint (Compl. ¶101).
  • Independent Claim 1 recites a method comprising:
    • Acquiring images of a first view of an object from image sensors.
    • Identifying features in the images.
    • Determining the number of additional views needed based on factors including the number of sensors, identified features, and known invariant physical relationships, to ensure a solvable system of equations.
    • Acquiring the additional views.
    • Employing the consistency of physical relationships between features to set up a system of equations.
    • Computationally solving the system of equations.

Multi-Patent Capsule: U.S. Patent No. 8,095,237 - "Method and Apparatus for Single Image 3D Vision Guided Robotics," Issued January 10, 2012

  • Technology Synopsis: This patent, a continuation-in-part of the application leading to the ’755 patent, describes a method for three-dimensional pose estimation using a single camera mounted on a robot (’237 Patent, col. 1:5-9). The method determines an "object space-to-camera space transformation" for a target object based on features located in a single captured image and an algorithm that employs a known or determinable physical relationship between those features (’237 Patent, Abstract).
  • Asserted Claims: Independent claim 1 and dependent claims 2-5, 7-17, and 20-28 (Compl. ¶¶111, 112).
  • Accused Features: The complaint alleges that ABB's Integrated Vision, YuMi, and 3DQI products and associated software perform the claimed method of 3D pose estimation (Compl. ¶112).

III. The Accused Instrumentality

Product Identification

  • The complaint identifies several accused product lines: ABB's Integrated Vision system, YuMi collaborative robots, 3DQI (3D Quality Inspection) products, and FlexLoader products (Compl. ¶¶50, 78).

Functionality and Market Context

  • The accused instrumentalities are components of ABB's broader "robotic and discrete automation systems" portfolio (Compl. ¶49). The complaint alleges their relevant functionality involves machine vision for robotic guidance. The Integrated Vision system is described as a "powerful smart camera system" for "vision-guided robotics applications" (Compl. ¶56). The YuMi products are alleged to use "camera-based part location" (Compl. ¶62), and the 3DQI products offer "robotic 3D quality inspection" using "reference points control" (Compl. ¶74). Figure 2 from the complaint, though depicting the plaintiff's own software, illustrates the type of feature-based object training ("anchor" features and "GeoPatterns") that is central to the technology in dispute (Compl. ¶38).
  • The complaint frames these products as commercially important offerings in the industrial and collaborative robotics market (Compl. ¶49).

IV. Analysis of Infringement Allegations

The complaint does not provide sufficient detail for a full claim chart analysis. The infringement allegations are pleaded at a high level, stating that the accused product families "implement or contain each and every element" of the asserted claims and referring to infringement claim charts served as separate exhibits which were not attached to the complaint (Compl. ¶¶93, 102).

The narrative infringement theory for the ’755 patent is that ABB's Integrated Vision, YuMi, and 3DQI products perform the patented method of 3D object handling using a single camera, including the steps of calibration, feature teaching, and object finding as recited in the claims (Compl. ¶¶92-93). For the ’814 patent, the theory is that ABB's Integrated Vision, YuMi, FlexLoader, and 3DQI products perform the patented method of machine vision, including determining the necessary number of views to solve for an object's pose using invariant physical relationships (Compl. ¶¶101-102). The complaint provides visual evidence, such as an example of a calibration grid from Plaintiff's software in Figure 1, to explain the underlying technology of camera calibration, a key step in the asserted claims (Compl. ¶36).

  • Identified Points of Contention:
    • Factual Question: A central question will be evidentiary. Given the lack of technical detail in the complaint about the accused products' specific software architecture, the dispute will likely focus on whether discovery reveals that ABB's systems perform the specific, multi-step computational processes recited in the claims. For example, does the accused "Integrated Vision" system (Compl. ¶53) perform the claimed transformations between an "Object Space" and "Camera Space" as required by claim 8 of the ’755 patent?
    • Technical Question: For the ’814 patent, a key technical question will be whether ABB's systems perform the claimed step of "determining a number of additional views to be obtained" based on an analysis of known physical relationships (Compl. ¶101). The plaintiff will need to show that the accused systems contain this specific "intelligent view planning" logic, rather than simply using a pre-set or fixed number of views. Figure 3, showing an example of software output calculating an object's position, highlights the type of computational result the patented systems are designed to achieve (Compl. ¶39).

V. Key Claim Terms for Construction

'755 Patent

  • The Term: "Object Space" (from claim 8)
  • Context and Importance: This term is a cornerstone of the claimed method, which relies on a framework of defined coordinate systems and the transformations between them. Infringement will depend on whether ABB's systems utilize a computationally equivalent architecture. Practitioners may focus on this term to determine if the patent's specific framework can be mapped onto the accused systems' operations.
  • Intrinsic Evidence for Interpretation:
    • Evidence for a Broader Interpretation: The specification provides a functional definition: "a reference frame defined with respect to, and therefore rigid to, the object 18" (’755 Patent, col. 4:10-12). This language could support a construction covering any system that establishes a coordinate frame fixed to the target object.
    • Evidence for a Narrower Interpretation: The specification also describes a specific method of its creation: "defining an 'Object Space' aligned with the 'Training Space' but connected to the object and transposing the 3D coordinates of the features into the 'Object Space'" (’755 Patent, col. 2:6-10). This could support an argument that the term is limited to a coordinate system created through this specific transposition process.

'814 Patent

  • The Term: "determining a number of additional views to be obtained" (from claim 1)
  • Context and Importance: This limitation appears central to the patent's purported advance over the prior art—the ability to dynamically plan the data acquisition process based on incomplete information. The infringement analysis will likely turn on whether ABB's systems perform such a "determination" or use a different, less adaptive approach.
  • Intrinsic Evidence for Interpretation:
    • Evidence for a Broader Interpretation: A plaintiff may argue the term covers any process that adaptively acquires more views when initial data is insufficient to solve for the object's pose. The claim language requires the determination to be based "at least in part" on the available features and relationships, suggesting the process need not be exclusively mathematical.
    • Evidence for a Narrower Interpretation: The claim requires the determination to be "sufficient to provide a system of equations and unknowns where the number of unknowns is not greater than the number of equations" (’814 Patent, col. 24:14-18). A defendant may argue this requires an explicit, formulaic calculation of the minimum views needed, as illustrated by equations in the specification (e.g., Equation 0 at col. 11:50), rather than a more general or heuristic approach.

VI. Other Allegations

  • Indirect Infringement: The complaint alleges induced infringement, stating that ABB provides user manuals, online tutorials through the "ABB Robotics Academy," and installation support services that instruct and encourage customers to operate the accused systems in a manner that directly infringes the patents (Compl. ¶¶80-83, 95, 104).
  • Willful Infringement: The willfulness claim is based on alleged knowledge from multiple sources. For pre-suit conduct, the complaint points to the 2006-2008 licensing agreement with Plaintiff's predecessor for the commercial embodiment of the patents, and to marketing materials from that era referencing "patented single camera 3D technology" (Compl. ¶¶87, 96, 105). For post-suit conduct, knowledge is alleged based on infringement contentions served in a prior lawsuit between the parties in November 2023 (Compl. ¶¶97, 106, 117).

VII. Analyst’s Conclusion: Key Questions for the Case

  • A core issue will be one of evidentiary proof: The complaint establishes an extensive history between the parties but provides minimal technical detail on how the accused products operate. A central question for the litigation will be whether the plaintiff, through discovery, can produce evidence demonstrating that ABB's varied automation systems perform the specific, multi-step computational architectures recited in the asserted claims.
  • A second key issue will be one of definitional scope: The case may turn on whether key claim terms are construed broadly or narrowly. For example, can the '814 patent's requirement of "determining a number of additional views" be read on any adaptive view-taking process, or is it limited to the explicit mathematical determination detailed in the specification? The resolution of such questions will be critical in defining the line between infringement and non-infringing design.
  • A final key question will be one of intent: Given the parties' long and allegedly contentious history, including a prior licensing relationship and previous litigation, the willfulness allegations will be a significant focus. A primary question for the fact-finder will be whether Defendant's alleged conduct, in light of its purported knowledge of the patents dating back to 2006, rises to the level of egregious behavior that would warrant enhanced damages.