Image Zoom
Image Zoom

Object Detection and Tracking with Event Vision-Based Sensors

Event Vision-Based (EVB) sensing offers accelerated sensing speeds, improved operation in unpredictable lighting conditions, and decreased communication demands compared to frame-based sensors.¹ This application showcases an integrated EVB sensor via MIPI to the AMD Kria™ KV260 Vision AI Starter Kit, executing object detection and tracking for streamlined, end-to-end pipelined acceleration. The application includes a GitHub reference design, empowering customers to customize it to suit their specific needs.

Last Updated: Jul 23, 2024
AMD Tool Version: 2022.2

Evaluate Free

Contact LogicTronix

Product Features

This Kria KV260 Vision AI Starter Kit application provides easy bring-up steps for event camera interfacing and ML inferencing on event data streams. It also has a GitHub page that allows you to modify the pipeline to create your own customized applications.

Required Hardware

Kria™ KV260 Vision AI Starter Kit

Recommended Accessories:

Kria KV260 Basic Accessory Pack

Power supply & adapter, Ethernet cable, USB cable accessories, and DP or HDMI™ monitor.

View Product

PROPHESEE Metavision® Starter Kit

Stacked HD event-based vision sensor realized in collaboration between Sony® and PROPHESEE.

View Product

Partner Information

LogicTronix Technologies

LogicTronix is an FPGA design and machine learning acceleration company and AMD adaptive computing partner (ACP) that offers "vision edge AI" solutions, IP cores, and design services based on AMD FPGAs, adaptive SoCs, and technologies focusing on smart city, surveillance, Industry 4.0, medical, and automotive applications. LogicTronix provides a diverse range of IP cores targeted for computer vision, machine learning, and cryptographic hashing. For vision edge AI solutions, IP cores, and design service inquiries, please visit the LogicTronix ACP partner page.

Learn More

PROPHESEE

PROPHESEE is the inventor of the world’s most advanced neuromorphic vision systems.¹ The company developed a breakthrough event-based vision approach to machine vision. This new vision category allows for significant reductions of power, latency, and data processing requirements¹ to reveal what was invisible to traditional frame-based sensors until now. PROPHESEE’s patented Metavision sensors and algorithms mimic how the human eye and brain work to dramatically improve efficiency in areas such as autonomous vehicles, industrial automation, IoT, mobile, and AR/VR. PROPHESEE is based in Paris, with offices in Grenoble, Shanghai, Tokyo, and Silicon Valley. 

Learn More

white pearl gradient medium color divider

Support & Resources

Resources

Kria SOM SmartCam with Raspberry Pi Product Brief: PROPHESEE Metavision Starter Kit with AMD Kria KV260 Starter Kit PROPHESEE ML Training Tutorial with the Yolov8 Network for Event Data

Frequently Asked Questions

Each pixel reports when it detects a relative change in the illumination intensity that is above or below a defined percentage of the previous intensity. Instead of capturing images at a fixed time interval, the event camera senses the changes in brightness at a per-pixel level asynchronously and creates event data streams. Learn more about how event vision works here.

Event camera datasets are provided by several organizations. Visit the PROPHESEE dataset page, or go to the DSEC dataset or the eTraM dataset pages. If you have an event camera, you can capture and create a custom dataset based on its data streams.

If the event camera is available with a MIPI interface, then the Kria SOM SmartCam design with Raspberry Pi example can provide general guidance on creating AMD Vivado and AMD Vitis designs. Since the event camera generates event data differently from an image-based frame camera, the Vivado IP pipeline will consist of a MIPI-DMA-PS (DDR) pipeline but does not require demosaic, gamma, and color correction IP cores in between. For a USB-based event camera, you can set up the necessary Linux® driver for the USB-based event camera and start capturing and working with the camera. Here is a GitHub link for an application design. For further customization, please click the Contact LogicTronix button above.

An event camera is triggered by an event, so unlike an RGB image camera, it generates data streams when there is movement at the focused or capturing region. Therefore, overall processing bandwidth requirements will be lower than the RGB data and camera. An event camera can also perform at higher speeds than a traditional image camera.¹ More information about the advantages of event-based cameras and vision is available here.

This tutorial showcases running event-based ML training and inferencing that is similar to RGB image-based cameras. You can also visit the PROPHESEE Yolo tutorial page for training/inferencing of a Yolo network for event-based data. Other tutorials for training and inferencing ML algorithms or networks on event camera data are also available.

Footnotes

All performance benefits and/or time savings claims are provided by PROPHESEE and/or LogicTronix Technologies and have not been independently tested or verified by AMD. Performance benefits and time savings are impacted by a variety of variables. Results herein are specific to PROPHESEE and/or LogicTronix Technologies and may not be typical. GD-181

_{Certain AMD technologies may require third-party enablement or activation. Supported features may vary by operating system. Please confirm with system manufacturer for specific features. No technology or product can be completely secure.}

Data Center

Business Systems

Personal & Gaming

Embedded

Resources

GPU Accelerators

Adaptive Accelerators

DPU Accelerators

Ethernet Adapters

Workstations

Desktops

Laptops

Resources

Adaptive SoCs & FPGAs

System-on-Modules (SOMs)

Technologies

Resources

Evaluation Boards & Kits

Processor Tools

Graphics Tools & Apps

Adaptive SoC & FPGA Tools

Intellectual Property & Apps

GPU Accelerator Tools & Apps

Overview

For Data Center & Cloud

For Edge & Endpoints

For Developers

Industries

Industries

Industries

Industries

Industries

Workloads

Gaming

Systems

Technologies

Resources

EPYC Processors

Radeon Graphics & AMD Chipsets

Adaptive SoCs & FPGAs

Alveo Accelerators & Kria SOMs

Ryzen Processors

Ethernet Adapters

Overview

Processors

Accelerators

Adaptive SoCs, FPGAs, & SOMs

Graphics

Overview

Resources by Product

Resources by Type

About Our Partners

AMD Global Support

Processors & Graphics

Accelerators

Adaptive SoCs & FPGAs

Gaming & Personal Computing

Adaptive & Embedded Computing

Get AMD Fan Gear

Shop Our Retail Partners

Object Detection and Tracking with Event Vision-Based Sensors

Product Features

Required Hardware

Recommended Accessories:

Partner Information

Support & Resources

Resources

Frequently Asked Questions

How does an event camera work?

How do I acquire event camera datasets?

How do I design with the AMD Vivado™ Design Suite or AMD Vitis™ unified software platform for an event camera?

What are the advantages of an event camera over a typical RGB camera?

How do I run a machine learning model using an event camera?