DCT
3:23-cv-00673
Clear Imaging Reasearch LLC v. Apple Inc
I. Executive Summary and Procedural Information
- Parties & Counsel:
- Plaintiff: Clear Imaging Research, LLC (Delaware)
- Defendant: Apple Inc. (California)
- Plaintiff’s Counsel: McKool Smith, P.C.
- Case Identification: 3:23-cv-00673, S.D. Cal., 04/14/2023
- Venue Allegations: Plaintiff alleges venue is proper in the Southern District of California because Apple is incorporated in California, has committed acts of infringement in the district through sales at local Apple Stores, and maintains a regular and established place of business in San Diego, including research and development facilities focused on the accused camera technology.
- Core Dispute: Plaintiff alleges that Defendant’s iPhone and iPad products infringe seven patents related to digital image and video stabilization and processing.
- Technical Context: The technology involves software and hardware-based methods for correcting motion blur, stabilizing video, and creating computational depth-of-field effects in digital cameras, which are critical features in the competitive smartphone market.
- Key Procedural History: The complaint alleges a lengthy history of communication between the parties regarding the patented technology, beginning in 2017. It also references a prior successful licensing campaign that included Defendant's competitor, Samsung. Notably, the complaint asserts that in 2014, the U.S. Patent and Trademark Office rejected an Apple patent application for similar technology based on the teachings of the patents' named inventor, which may be presented as evidence of early knowledge.
Case Timeline
| Date | Event |
|---|---|
| 2004-03-25 | Earliest Patent Priority Date |
| 2014-01-01 | Apple allegedly sought to patent similar technology |
| 2016-09-16 | Apple releases iPhone 7 Plus with Portrait Mode |
| 2017-07-25 | Plaintiff sent first notice letter to Defendant |
| 2018-01-02 | U.S. Patent No. 9,860,450 Issues |
| 2019-01-01 | U.S. Patent No. 10,171,740 Issues |
| 2019-08-13 | U.S. Patent No. 10,382,689 Issues |
| 2020-10-23 | Apple releases iPhone 12 Pro Max with Sensor-Shift OIS |
| 2022-09-27 | U.S. Patent No. 11,457,149 Issues |
| 2022-11-01 | U.S. Patent No. 11,490,015 Issues |
| 2023-04-11 | U.S. Patent No. 11,627,254 Issues |
| 2023-04-11 | U.S. Patent No. 11,627,391 Issues |
| 2023-04-14 | Complaint Filing Date |
II. Technology and Patent(s)-in-Suit Analysis
U.S. Patent No. 9,860,450 - "Method and Apparatus to Correct Digital Video to Counteract Effect of Camera Shake"
The Invention Explained
- Problem Addressed: The patent’s background section, along with the complaint, identifies the problem of motion blur in digital photography and video caused by camera movement or "shake" (Compl. ¶53; ’450 Patent, col. 1:40-52). Prior art solutions involved electro-mechanical devices built into high-quality lenses, which were described as expensive, heavy, less sturdy, and potentially compromising to image quality, making them unsuitable for lightweight devices (Compl. ¶53; ’450 Patent, col. 2:6-14).
- The Patented Solution: The invention proposes a digital processing solution where one or more motion sensors (e.g., accelerometers, gyroscopes) detect the motion of the imaging device during video capture ('450 Patent, Abstract). A processor uses this motion data to calculate vertical and horizontal shifts for the video frames and then modifies the images to counteract the shake, resulting in a stabilized final video ('450 Patent, Abstract). This software-based correction aims to provide stabilization without the drawbacks of mechanical lens systems ('450 Patent, col. 2:15-22).
- Technical Importance: This approach provided a path to implement video stabilization in compact, lightweight, and less expensive devices like smartphones, where bulky mechanical lens systems were considered infeasible (Compl. ¶53).
Key Claims at a Glance
- The complaint asserts independent claim 14 (Compl. ¶70).
- Claim 14 of the ’450 Patent: An imaging device comprising:
- an image sensor configured to capture a sequence of images, wherein the sequence of images comprise a video, and store the images in a memory;
- one or more motion sensors configured to detect motion information for one or more images of the sequence of images, wherein the motion information represents motion of the imaging device during capturing... and store the motion information in the memory synchronously with the storing of the one or more images;
- a processor configured to determine a vertical shift value and a horizontal shift value for one or more images... based at least in part on the motion information;
- the processor is further configured to modify one or more images of the sequence of images based at least in part on the vertical and the horizontal shift values;
- the processor is further configured to combine the modified images to obtain a final video; and
- wherein the memory is further configured to store the final video.
- The complaint reserves the right to assert additional claims (Compl. ¶27, footnote 27).
U.S. Patent No. 10,171,740 - "Method and Apparatus to Correct Blur in All or Part of a Digital Image By Combining Plurality of Images"
The Invention Explained
- Problem Addressed: The patent addresses the technical challenge of creating a photograph with a sharp, clear subject against a blurred background—an effect known as shallow depth-of-field or "bokeh" (’740 Patent, col. 13:43-52). Achieving this effect with traditional cameras typically requires expensive, heavy lenses with wide apertures, which are not practical for compact devices like smartphones (Compl. ¶51; ’740 Patent, col. 2:20-33).
- The Patented Solution: The invention describes a computational method where the device captures a plurality of images and combines them to achieve the desired effect ('740 Patent, Abstract). The system allows for a subject to be designated—either by user input or automatically—to be kept blur-free. The processor then combines the captured images to generate a final, composite image where the designated subject is clear and sharp, while the rest of the image (the background) is blurred ('740 Patent, col. 12:51-67).
- Technical Importance: This technology underpins features like "Portrait Mode" in smartphones, enabling them to computationally replicate an aesthetic effect previously achievable only with high-end DSLR cameras and specialized lenses (Compl. ¶60-61).
Key Claims at a Glance
- The complaint asserts independent claim 20 (Compl. ¶79).
- Claim 20 of the ’740 Patent: An imaging device for capturing and processing images, comprising:
- a user interface configured to display an image, wherein the image is a preview of the field of view of the device, and wherein the image includes a first subject and a second subject;
- a processor configured to receive user input designating the first subject in the image to be kept blur free;
- a memory configured to store a plurality of images captured by the device, wherein the plurality of images include the first subject and the second subject;
- the processor further configured to combine the plurality of images to obtain a combined image, such that: the combined image includes the first subject and the second subject, the first subject in the combined image is blur free, and the second subject in the combined image is blurred compared to the first subject;
- the user interface further configured to display the combined image; and
- a memory configured to store the combined image.
- The complaint reserves the right to assert additional claims (Compl. ¶27, footnote 27).
U.S. Patent No. 10,382,689 - "Method and Apparatus for Capturing Stabilized Video in an Imaging Device"
- Technology Synopsis: This patent is related to the ’450 patent and addresses video stabilization. The invention describes an apparatus that uses motion sensors to generate motion data, a video image processor to calculate and apply shift amounts to a sequence of images based on that data, and a movable lens element that can also be used to compensate for motion (Compl. ¶53-54; ’689 Patent, Abstract). The result is a stabilized video that is stored in memory.
- Asserted Claims: The complaint asserts at least independent claim 1 (Compl. ¶89).
- Accused Features: The accused features are the video stabilization functionalities in the Accused Products, such as Cinematic Video Stabilization and Action Mode, which allegedly use motion sensors and processing to stabilize video (Compl. ¶91, 94).
U.S. Patent No. 11,457,149 - "Method and Apparatus for Capturing Digital Video"
- Technology Synopsis: This patent is related to the ’740 patent and addresses computational photography for creating depth-of-field effects. The invention describes receiving a sequence of digital images, processing them to determine a designated subject and background by taking the lens's focal length into account, and generating a final image where the subject is clear and the background is blurred (Compl. ¶51; ’149 Patent, Abstract).
- Asserted Claims: The complaint asserts at least independent claim 21 (Compl. ¶98).
- Accused Features: The accused features are the "Portrait Mode" and "Cinematic Mode" functionalities, which allegedly capture and process multiple images to create a final image or video with a sharp subject and a blurred background (Compl. ¶100, 103).
U.S. Patent No. 11,490,015 - "Method and Apparatus for Capturing Digital Video"
- Technology Synopsis: This patent is related to the ’149 patent and also addresses creating depth-of-field effects, but with a focus on video. The invention describes processing a plurality of images based on focal length and relative shift to generate a final video where a designated subject is blur-free and the background is blurred (Compl. ¶57; ’015 Patent, Abstract).
- Asserted Claims: The complaint asserts at least independent claim 21 (Compl. ¶106).
- Accused Features: The accused feature is primarily Apple's "Cinematic Mode," which provides a shallow depth-of-field effect for video, allegedly by processing multiple images to distinguish a subject from its background and apply a blur computationally (Compl. ¶107, 109).
U.S. Patent No. 11,627,254 - "Method and Apparatus for Capturing Digital Video"
- Technology Synopsis: This patent addresses image stabilization through physical movement of the image sensor itself. The invention describes using motion sensors to generate motion information, which then causes image sensor adjusters to physically move the sensor inside the apparatus to counteract the detected motion (Compl. ¶55; ’254 Patent, Abstract). The invention also covers combining multiple images taken in rapid succession to create a corrected final image.
- Asserted Claims: The complaint asserts at least independent claim 11 (Compl. ¶116).
- Accused Features: The accused features include Apple's "Sensor-Shift" optical image stabilization, as well as multi-frame processing techniques like "Deep Fusion" and "Night Mode" (Compl. ¶117, 120, 122).
U.S. Patent No. 11,627,391 - "Method and Apparatus for Capturing Digital Video"
- Technology Synopsis: This patent is related to the ’015 patent and also addresses creating depth-of-field effects in video. The invention describes reading image data, designating a subject and background based on processing two or more images, and then obtaining a sequence of modified images where the main subject is blur-free and the background is blurred for use in a final video (Compl. ¶57; ’391 Patent, Abstract).
- Asserted Claims: The complaint asserts at least independent claim 8 (Compl. ¶126).
- Accused Features: The accused feature is Apple's "Cinematic Mode," which allegedly creates a video with a sharp subject and blurred background by processing sequences of images (Compl. ¶128, 132).
III. The Accused Instrumentality
Product Identification
The complaint accuses a wide range of Apple products, including iPhones from the iPhone 6 model forward and various generations of the iPad, iPad Pro, iPad Air, and iPad mini (Compl. ¶3). Collectively, these are referred to as the "Accused Products."
Functionality and Market Context
The infringement allegations target specific, high-profile camera functionalities that are central to Apple's product marketing. These functionalities include:
- Video Stabilization: Features such as standard Video Image Stabilization, "Cinematic Video Stabilization," and "Action Mode" are accused of infringing patents related to correcting for camera shake (Compl. ¶71, 89). The complaint provides a screenshot from an Apple event demonstrating "Cinematic video stabilization" (Compl. p. 26). This functionality uses on-board motion sensors (gyroscopes, accelerometers) and a dedicated motion coprocessor to detect and compensate for device movement, producing smoother video (Compl. ¶71, 91).
- Computational Depth-of-Field: "Portrait Mode" for still images and "Cinematic Mode" for video are accused of infringing patents related to creating a shallow depth-of-field effect (Compl. ¶79, 107). These features use data from single or multiple cameras to computationally generate a depth map, allowing the software to distinguish a subject from its background and apply an artificial blur to the background, simulating the "bokeh" effect of a professional camera (Compl. ¶61). The complaint includes a graphic illustrating how Apple's software uses "Face detection" and "Depth mapping" to achieve this effect (Compl. pp. 39-40).
- Sensor-Shift Stabilization: Apple's "Sensor-shift" optical image stabilization, first introduced in the iPhone 12 Pro Max, is accused of infringing patents directed to moving the image sensor itself to counteract motion (Compl. ¶117). This is presented as a high-end stabilization technique that moves the sensor rather than the heavier lens elements, allowing for more precise control (Compl. ¶79).
- Multi-Frame Image Processing: Features like "Deep Fusion," "Night Mode," and "Photographic Styles" are accused of infringing patents that involve capturing and combining a sequence of images to produce a single, enhanced final image (Compl. ¶120). These features work by capturing multiple frames at different exposures before and after the shutter is pressed and using a neural engine to fuse them, optimizing for detail and low noise (Compl. p. 84).
IV. Analysis of Infringement Allegations
U.S. Patent No. 9,860,450 - Infringement Allegations
| Claim Element (from Independent Claim 14) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| an image sensor configured to capture a sequence of images, wherein the sequence of images comprise a video, and store the images in a memory; | The Accused Products contain an image sensor that captures video as a sequence of images and stores them in memory. | ¶70 | col. 4:60-65 |
| one or more motion sensors configured to detect motion information... and store the motion information in the memory synchronously with the storing of the one or more images. | The Accused Products include motion sensors (accelerometer, gyroscope) and a motion coprocessor that detect device motion and store this information synchronously with video frames. | ¶71 | col. 10:20-29 |
| a processor configured to determine a vertical shift value and a horizontal shift value for one or more images... based at least in part on the motion information; | Apple’s AVFoundation and CoreMotion software frameworks allegedly use the motion data to determine vertical and horizontal shifts as part of its digital video stabilization. | ¶72 | col. 10:49-55 |
| the processor is further configured to modify one or more images... based at least in part on the vertical and the horizontal shift values; | Apple's digital video stabilization functionality allegedly modifies the captured images to compensate for the calculated vertical and horizontal shifts. | ¶72 | col. 11:1-5 |
| the processor is further configured to combine the modified images to obtain a final video; and wherein the memory is further configured to store the final video. | The processor in the Accused Products combines the modified frames to create the final, stabilized video, which is then stored in the device's memory. | ¶73 | col. 11:6-12 |
Identified Points of Contention
- Scope Questions: The interpretation of "synchronously" in the claim will be a point of focus. The dispute may center on whether Apple's method of associating motion data with video frames meets the specific temporal relationship required by the claim language as understood in light of the patent's specification.
- Technical Questions: A key evidentiary question is whether Apple's stabilization algorithms, such as "GyroVideoStabilization," explicitly "determine a vertical shift value and a horizontal shift value" and "modify" the images based on those specific values, as required by the claim. The defense may argue that its proprietary stabilization methods operate on different principles that do not map onto these specific claimed steps.
U.S. Patent No. 10,171,740 - Infringement Allegations
| Claim Element (from Independent Claim 20) | Alleged Infringing Functionality | Complaint Citation | Patent Citation |
|---|---|---|---|
| a user interface configured to display an image, wherein the image is a preview... and wherein the image includes a first subject and a second subject; | The Accused Products' camera application displays a live preview on the screen showing people (first subject) and their surroundings (second subject). | ¶79 | col. 12:20-25 |
| a processor configured to receive user input designating the first subject in the image to be kept blur free; | The user interface allows a user to select "Portrait" mode, which designates the primary subject to be kept sharp. The complaint alleges this constitutes the required user input. | ¶79, 81 | col. 12:26-29 |
| a memory configured to store a plurality of images captured by the device, wherein the plurality of images include the first subject and the second subject; | The Accused Products' Portrait Mode captures multiple images, including from two different cameras, which are stored in memory for processing. | ¶80 | col. 12:30-34 |
| the processor further configured to combine the plurality of images to obtain a combined image, such that... the first subject... is blur free, and the second subject... is blurred compared to the first subject; | The processor uses data from the multiple captured images to generate a depth map and computationally apply a blur to the background, creating a final combined image where the subject is sharp. | ¶81 | col. 12:35-43 |
| the user interface further configured to display the combined image; and a memory configured to store the combined image. | The final Portrait Mode photo is displayed on the device's screen and saved to the device's memory (e.g., the Photos app). | ¶82, 83 | col. 12:44-48 |
Identified Points of Contention
- Scope Questions: A central dispute will likely be the meaning of "receive user input designating the first subject." Apple's Portrait Mode often uses machine learning to automatically detect a person as the subject without explicit user interaction. The question for the court will be whether activating the mode itself, or the software's automatic detection, satisfies this limitation, or if it requires a specific user action like tapping on the subject. The complaint presents a graphic of "Face detection" which suggests Apple's automated process (Compl. p. 39).
- Technical Questions: Does Apple's process of using two cameras to create a depth map and then applying a blur filter constitute "combin[ing] the plurality of images to obtain a combined image" as that phrase is used in the patent? The defense may argue that its method is a fundamentally different technical approach from the "combining" described in the '740 patent's specification.
V. Key Claim Terms for Construction
Term for Construction: "store the motion information in the memory synchronously with the storing of the one or more images" (from the ’450 Patent)
- Context and Importance: This term is critical for determining infringement by the video stabilization features. The precise timing relationship between capturing motion data and capturing video frames is central. Practitioners may focus on this term because if "synchronously" is construed to require a very strict, simultaneous hardware-level event, it might not read on a software-based system where motion data is merely timestamped and later associated with a corresponding video frame.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The specification's general description of using motion sensor data to correct video frames may support a construction where any reliable association between a frame and the motion that occurred during its capture is sufficient ('450 Patent, Abstract).
- Evidence for a Narrower Interpretation: The defense may point to specific embodiments or figures in the patent that imply a tighter, more direct hardware link between the sensors and memory storage, arguing for a more restrictive definition of the term.
Term for Construction: "receive user input designating the first subject in the image to be kept blur free" (from the ’740 Patent)
- Context and Importance: This term's construction is fundamental to whether Apple's automated "Portrait Mode" infringes. If the term is construed to require a manual, explicit user selection of the subject (e.g., tapping on a face), then a mode that automatically selects the subject may not literally infringe. Practitioners may focus on this term because it is a clear potential line of non-infringement for Apple's AI-driven features.
- Intrinsic Evidence for Interpretation:
- Evidence for a Broader Interpretation: The patent states that a "user can indicate, such as by an indicator in a viewfinder, which object in the field of view is the designated subject to be kept blur-free" ('740 Patent, col. 11:5-9). Plaintiff may argue that selecting "Portrait Mode" from the camera UI is itself the "user input" that designates whatever the algorithm subsequently identifies as the subject.
- Evidence for a Narrower Interpretation: The claim language "designating the first subject" could be interpreted to mean the user input must itself perform the designation, not merely activate a mode that performs the designation. The defense could argue that simply selecting a mode does not constitute "designating the first subject."
VI. Other Allegations
Indirect Infringement
The complaint pleads inducement for all seven patents. It alleges that Apple actively encourages infringement by providing user manuals, online support documents, marketing materials, and software interfaces (e.g., the Camera app) that instruct and enable users to operate the Accused Products in an infringing manner, for example by using Portrait Mode or video stabilization features (Compl. ¶75, 85, 94, 103, 112, 122, 132).
Willful Infringement
The complaint alleges willful infringement based on both pre- and post-suit knowledge. Pre-suit knowledge is alleged to have been established through nearly six years of licensing discussions, meetings, and the provision of claim charts to Apple beginning in 2017 (Compl. ¶14-28). The complaint further alleges knowledge dating to 2014, when the USPTO cited technology from the patents' inventor against Apple's own patent application for video stabilization (Compl. ¶6).
VII. Analyst’s Conclusion: Key Questions for the Case
- A core issue will be one of definitional scope: Can claim terms like "receive user input designating the first subject" be construed to cover modern, AI-driven camera features that automatically detect and select subjects without explicit manual designation? The outcome of this claim construction battle will likely determine the viability of infringement allegations against Apple's "Portrait Mode."
- A second central question is one of technical implementation: Do Apple's sophisticated, multi-step computational photography processes—such as generating depth maps from dual-camera parallax and using neural engines for multi-frame fusion—perform the same function in substantially the same way as the methods for "combining" images and "modifying" video frames described and claimed in the patents?
- A final key issue will be willfulness: Given the extensive allegations of pre-suit communications and Apple's own patent prosecution history, the court will have to evaluate whether Apple's conduct was objectively reckless. This will be a fact-intensive inquiry with significant implications for potential enhanced damages if infringement is found.