DCT
1:22-cv-01257
RoboticVISIONTech Inc v. ABB Inc
I. Executive Summary and Procedural Information
- Parties & Counsel:
- Plaintiff: RoboticVISIONTech, Inc. (Delaware)
- Defendant: ABB Inc. (Delaware)
- Plaintiff’s Counsel: Shaw Keller LLP
- Case Identification: 1:22-cv-01257, D. Del., 09/22/2022
- Venue Allegations: Venue is alleged to be proper in the District of Delaware because Defendant is a Delaware corporation and is therefore a resident of the district.
- Core Dispute: Plaintiff alleges that Defendant’s FlexVision 3D robotic vision software infringes three patents, misappropriates trade secrets, and infringes copyrights related to 3D object location and guidance using 2D imaging.
- Technical Context: The technology at issue is 3D vision-guided robotics, a critical component in industrial automation that enables robots to identify, locate, and manipulate objects in a three-dimensional workspace for manufacturing and assembly tasks.
- Key Procedural History: The complaint outlines a complex history between the parties, beginning with a 2006 licensing agreement between Defendant and Plaintiff’s predecessor-in-interest for the software embodying the asserted patents. Plaintiff alleges that after it acquired the patent portfolio in 2010, Defendant hired one of Plaintiff's chief scientists—a named inventor on the asserted patents—and subsequently launched the accused competing product in 2015.
Case Timeline
| Date | Event |
|---|---|
| 2002-05-24 | Priority Date for ’755 and ’237 Patents |
| 2004-07-14 | Priority Date for ’814 Patent |
| 2004-11-09 | ’755 Patent Issued |
| 2006-05-01 | Braintech (Plaintiff's predecessor) and Defendant enter into partnership agreement |
| 2008-02-26 | ’814 Patent Issued |
| 2010-05-01 | Plaintiff acquires Braintech's assets, including patents-in-suit |
| 2010-10-01 | Defendant hires Plaintiff’s Chief Scientist, Dr. Remus Boca |
| 2012-01-10 | ’237 Patent Issued |
| 2015-01-01 | Defendant launches accused FlexVision product |
| 2022-09-22 | Complaint Filed |
II. Technology and Patent(s)-in-Suit Analysis
U.S. Patent No. 6,816,755 - Method and Apparatus for Single Camera 3D Vision Guided Robotics
The Invention Explained
- Problem Addressed: The patent’s background section describes a need for a 3D object positioning method for vision-guided robots that avoids the cost and complexity of prior art systems, which often required two or more cameras or specialized laser sensors to operate in three dimensions (’755 Patent, col. 1:11-30).
- The Patented Solution: The invention proposes a method for a robot to determine an object’s 3D pose using only a single, standard video camera mounted on the robot’s arm. The process involves three main steps: calibrating the camera to the robot, teaching the system by identifying at least six features on the object from a 2D image, and then using those taught features to find the object in 3D space during operation (’755 Patent, Abstract; col. 2:52-58).
- Technical Importance: This approach sought to reduce the hardware cost and simplify the setup for 3D vision-guided robotics, thereby making the technology more accessible for a wider range of industrial automation applications (’755 Patent, col. 1:21-30).
Key Claims at a Glance
- The complaint asserts independent claim 8 (Compl. ¶144).
- Claim 8 recites a method with three main phases:
- Calibrating the camera: This involves finding the camera's intrinsic parameters and its position relative to the robot's tool ("hand-eye" calibration).
- Teaching the object features: This phase includes capturing an image, selecting at least six visible features, calculating their 3D position in an "Object Space," and computing various transformations to define an "Object Frame" that is sent to the robot.
- Carrying out object finding and positioning: This is the runtime phase where the robot locates the object using the taught features and performs its task.
- The complaint alleges infringement of claims 8-10, suggesting dependent claims may also be at issue (Compl. ¶145).
U.S. Patent No. 7,336,814 - Method and Apparatus for Machine-Vision
The Invention Explained
- Problem Addressed: The patent identifies drawbacks in prior systems that required a priori knowledge of the geometric relationships between all object features, often sourced from Computer-Aided Design (CAD) models that may not be available, or that required cumbersome inter-camera calibration (’814 Patent, col. 2:11-28).
- The Patented Solution: The invention provides a method to determine a 3D pose by exploiting "invariant physical relationships" (e.g., distances) between only some of the identified features. The method determines how many additional camera views are needed based on the number of features and known relationships, sets up a system of equations, and computationally solves it to find the 3D pose without complete geometric data (’814 Patent, Abstract; col. 2:36-54).
- Technical Importance: This technology increased the flexibility of 3D machine vision by reducing the reliance on complete CAD models, allowing for pose estimation of objects with only partially known or defined geometries (’814 Patent, col. 2:28-34).
Key Claims at a Glance
- The complaint asserts independent claims 1, 7, and 13 (Compl. ¶152).
- Claim 1 recites a method comprising the key steps of:
- Acquiring images of a first view of a training object.
- Identifying a number of features in the images.
- Determining a number of additional views to obtain based on the number of features and known "invariant physical relationship[s]" to create a solvable system of equations.
- Acquiring images from the additional views.
- Using the consistency of physical relationships between features across views to set up the system of equations.
- Automatically computationally solving the system of equations.
- The complaint alleges infringement of "one or more claims of the ’814 patent, including but not limited to claims 1, 7, and 13" (Compl. ¶153).
U.S. Patent No. 8,095,237 - Method and Apparatus for Single Image 3D Vision Guided Robotics
Technology Synopsis
This patent, a continuation-in-part of the application that led to the ’755 Patent, discloses a method for 3D pose estimation using a single camera mounted on a robot (’237 Patent, Abstract). The method focuses on determining an "object space-to-camera space transformation" for an object based on locating features within a single captured 2D image and employing a "known or determinable physical relationship between at least some of the located features" (’237 Patent, col. 12:54-67).
Asserted Claims
Independent claim 1 and dependent claims 2, 3, 7, and 10 are asserted (Compl. ¶161).
Accused Features
The complaint alleges that the FlexVision product infringes by capturing a 2D image, locating features within it, and determining the object's 3D pose (the transformation) using only that single image and the relationships between its features (Compl. ¶¶78, 86, 161).
III. The Accused Instrumentality
Product Identification
- Defendant's FlexVision 3D software product ("FlexVision") (Compl. ¶69).
Functionality and Market Context
- The complaint alleges FlexVision is a vision-guided robotic (VGR) software that enables robots to "precisely locate the grip points of a disoriented object within a 3D space" by using 2D images from one or more calibrated cameras (Compl. ¶¶74, 78). The software’s alleged functionality follows a process of camera calibration (producing intrinsic, extrinsic, and hand-eye data), object training via feature identification, and 3D pose estimation (Compl. ¶¶80-86). An image from the accused product's user manual shows a robot arm with a mounted camera, illustrating the hardware context in which the software operates (Compl. ¶80, Fig. 4).
- The complaint positions FlexVision as a key solution for powertrain automation in the automotive industry and alleges over 2,000 copies have been sold in the United States (Compl. ¶¶69, 71).
IV. Analysis of Infringement Allegations
U.S. Patent No. 6,816,755 Infringement Allegations
| Claim Element (from Independent Claim 8) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| i) calibrating the camera by finding a) the camera intrinsic parameters; b) the position of the camera relative to the tool of the robot ("hand-eye" calibration) | FlexVision performs calibration that produces three types of data: intrinsic, extrinsic, and hand-eye calibration data. | ¶80 | col. 4:21-29 |
| ii) teaching the object features by... a) putting the object in the field of view of the camera and capturing an image of the object | A camera mounted on the robot must capture at least one image of the part for FlexVision to perform its function. | ¶81 | col. 6:39-43 |
| b) selecting at least 6 visible features from the image | The FlexVision user manual allegedly instructs users to configure a minimum of four feature-detection tools but recommends "us[ing] at least 10 or 12 features" to adequately teach the robot. | ¶82 | col. 6:43-48 |
| c) calculating the 3D position in real world coordinates of said selected features inside a space connected to the object (“Object Space”) | FlexVision calculates the "real-world, three-dimensional location of these features" and generates a table of 3D coordinates. A screenshot shows this table of coordinates (Compl. ¶85, Fig. 6). | ¶¶84-85 | col. 6:48-52 |
| g) sending the computed “Object Frame” to the robot | The FlexVision user manual allegedly states that messaging is used "to send the calculated object pose back to the ABB Robot." | ¶87 | col. 6:62-64 |
| iii) carrying out object finding and positioning by... c) ...computing the object location as the transformation between the “Object Space” and “Camera Space” | FlexVision uses the identified features to estimate the pose of the part. The complaint includes a screenshot from FlexVision's interface showing multiple identified features on a part (Compl. ¶83, Fig. 5). | ¶86 | col. 8:41-47 |
U.S. Patent No. 7,336,814 Infringement Allegations
| Claim Element (from Independent Claim 1) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| acquiring a number of images of a first view of a training object from a number of image sensors | FlexVision uses one or more cameras to capture images of a target part. The complaint provides a screenshot from the user manual showing images captured by two separate cameras (Compl. ¶83, Fig. 5). | ¶¶81, 83 | col. 15:39-42 |
| identifying a number of features of the training object in the acquired at least one image of the first view | FlexVision uses feature detection tools to mark specific features of a part for 3D pose estimation. The interface allegedly shows 11 identified features on a part. | ¶82 | col. 15:43-45 |
| determining a number of additional views to be obtained based at least in part on... a type of the invariant physical relationship associated with the features, sufficient to provide a system of equations... | The complaint does not explicitly detail how FlexVision determines the number of views, but alleges that its overall process, which achieves the same result as Plaintiff's patented method, is evidence of infringement. | ¶¶98, 101 | col. 15:46-57 |
| employing at least one of a consistency of physical relationships between some of the identified features to set up the system of equations; and automatically computationally solving the system of equations | The complaint alleges that FlexVision achieves performance on par with Plaintiff's product only by incorporating its proprietary processes, implying the use of a similar computational method to determine the 3D pose from the identified features. | ¶¶99, 101 | col. 16:1-9 |
Identified Points of Contention
- Scope Questions: For the ’755 Patent, a potential dispute may arise over the "selecting at least 6 visible features" limitation. The complaint alleges the accused product's manual recommends 10-12 features but only requires a minimum of four. This raises the question of whether direct infringement occurs if a user follows the minimum requirement, and whether the product as a whole, when used as recommended, meets the claim limitation.
- Technical Questions: For the ’814 Patent, the complaint's allegations focus heavily on similarities in user interface and performance outcomes rather than the specific computational method claimed. A central technical question will be what evidence demonstrates that the accused product actually performs the claimed steps of determining the number of views needed and setting up and solving a "system of equations" based on "invariant physical relationships," as opposed to achieving a similar result through a different, non-infringing technical method.
V. Key Claim Terms for Construction
- The Term: "selecting at least 6 visible features" (’755 Patent, Claim 8)
- Context and Importance: This term is critical because the infringement allegation hinges on whether the accused product, which allegedly has a minimum requirement of four features but a recommendation of ten or more, meets this limitation. Its construction will likely determine whether infringement can be established based on the product's documented instructions and intended use.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: A party could argue that the phrase sets a baseline for functionality. The patent’s summary states the method uses "at least 6 visible features," which could be interpreted as the number required for the robust operation taught by the patent, a standard that a recommended use of "10 or 12 features" would meet (’755 Patent, col. 2:50-51).
- Evidence for a Narrower Interpretation: The claim language is numerically specific. The detailed description and flow chart in Figure 6 consistently refer to selecting "at least one anchor feature and a set of at least 5 other visible features," reinforcing the specific threshold of six total features (’755 Patent, col. 6:45-48). This could support a strict, literal interpretation where any operation with fewer than six features falls outside the claim scope.
VI. Other Allegations
- Indirect Infringement: The complaint alleges induced infringement, stating that Defendant provides customers with the FlexVision User Manual, which contains "explicit instructions for installing, configuring, and operating" the product in a manner that allegedly infringes the Asserted Patents (Compl. ¶104, 146).
- Willful Infringement: Willfulness is alleged based on pre-suit knowledge of the patents. The complaint claims Defendant was aware of the ’755 and ’814 patents since at least 2006 through a licensing agreement with Plaintiff's predecessor for software that embodied the patents (Compl. ¶108, 147). The complaint further alleges Defendant was aware of the ’237 patent upon its issuance in 2012, in part because Defendant had hired one of the patent's named inventors in 2010 (Compl. ¶164).
VII. Analyst’s Conclusion: Key Questions for the Case
The resolution of this dispute may depend on the court's findings on several central questions that bridge the case's patent, trade secret, and copyright claims.
- A core issue will be one of technological provenance and proof: Can Plaintiff demonstrate that the accused FlexVision software, which is based on a different third-party vision library (Cognex) than Plaintiff's product (Matrox), achieves its functionality by implementing the specific methods claimed in the patents? The complaint's narrative regarding the hiring of a key inventor and the extensive visual comparisons of the software interfaces, such as the annotated logic tree comparison, will be central to arguing that the similarities are not coincidental (Compl. ¶92, Fig. 9).
- A second key question will be one of operational scope: Does the accused product's functionality meet the "at least 6 visible features" limitation of the ’755 patent? The case may turn on whether the product's documented recommendation to use more than six features constitutes infringement, even if its stated minimum requirement is below that threshold.