1. Program Introduction
This solution is based on the STM32N6 main control platform paired with ST's VD66GY image sensor to achieve human posture monitoring.The system uses the VD66GY sensor to collect environmental image data, which is then efficiently analyzed and processed using the STM32N6 processor.
2. Introduction to Main Components
STM32N6
Based on the Arm Cortex-M55 microcontroller, equipped with the ST Neural-ART accelerator, H264 encoder, Neo-Chrom 2.5D graphics processor, 4.2MB continuous static random-access memory for real-time data processing and multitasking, dedicated embedded neural processing unit (600 GOPS NPU & 3 TOPS/W power consumption), image signal processor (ISP) (Parallel and MIPI CSI-2 camera module interface) supporting a 5 Mpixel camera and 30 FPS, with features including bad pixel correction, downsampling, black level adjustment, exposure control, demosaicing, column conversion, contrast adjustment, cropping, scaling, gamma correction, and pixel packing.
Engineers can assign machine learning inference tasks to the NPU, allowing the efficient Cortex-M processor, ISP, and GPU to focus on handling other system-level tasks or running user applications.This enables it to handle complex image recognition and processing tasks, such as object segmentation and localization, pose estimation, object classification, face/person detection, anomaly detection, and more.
The ST official website offers a variety of development kits and tools.It has its own discussion community and wiki, providing users with abundant resources and a free development environment.
Image Sensor - ST VD66GY
1.5M BSI sensor (1124x1364), configurable in both RGB and RGB-IR modes, with a dynamic range of up to 60dB and a pixel size of 2.61μm. These sensors perform exceptionally well in the visible light and near-infrared regions (940nm). In the 1124x1364 resolution format, they can capture up to 88 frames per second and offer two pixel matrix versions: RGB 2x2 core and RGB-NIR 4x4 core. The pixel structure of these devices minimizes crosstalk and maintains high MTF (Modulation Transfer Function) in both visible light and infrared regions.
ST also offers the VD66GY S-Board, which can be directly paired with the STM32N6 for development. It provides support from prototyping to mass production, not only saving resources but also accelerating the product launch timeline.
The VD66GY S-Board is a comprehensive multifunctional hardware kit designed for evaluating and integrating the VD66GY image sensor. It includes all the necessary hardware, from electronic components to optical devices, to perform thorough product evaluations or directly integrate the product into the rest of your system. The kit features an embedded VD66GY image sensor with an FFC output connector, an M12 lens holder, a default non-glued lens, and a ribbon cable. The non-glued lens included in the kit offers flexibility for focusing according to your needs or can simply be removed to use a different lens. The kit is equipped with an M12 lens interface, allowing customization with various M12 or smaller off-the-shelf lenses to suit your application. Its standard FFC connector enables the connection of cables with various lengths or pin configurations to adapt to your setup.
(Detailed information:VD66GY - Compact, high-sensitivity 1.53 MP color global shutter image sensor - STMicroelectronics）
3. Main Functions and Applications
The VD66GY image sensor collects front-facing image data, which is processed through the STM32N6 for inference tasks to identify and classify object types in the image, enabling corresponding functional designs. Leveraging the high-performance characteristics of the STM32N6 main controller and the VD66GY sensor, we can further expand its application scope and functionalities: anomaly detection in smart industries, event monitoring in smart homes, body data measurement in personal healthcare, collision detection for robots, environmental perception in automotive systems, and the development of personal electronic devices and sports equipment. This not only enhances the efficiency and quality of existing industrial and commercial applications but also extends to a broader range of intelligent technology applications, laying a solid foundation for future technological innovations.
It is also possible to leverage the edge computing and artificial intelligence technologies of STM32N6 (such as personnel detection, pose estimation, and gesture recognition) to enhance the range of product application services.
High-precision personnel detection over long distances in various environments: smart doorbells, room occupancy monitoring, alarm systems.
High-precision multi-posture estimation: behavior analysis, activity monitoring, fall detection.
Precise system control based on hand feature points: contactless human-machine interface and intelligent electronics industry.
►场景应用图
►产品实体图
►展示板照片
►方案方块图
►核心技术优势
STM32N6: • A microcontroller based on Arm Cortex-M55, equipped with the ST Neural-ART accelerator, H264 encoder, Neo-Chrom 2.5D graphics processor, and 4.2MB of continuous static random-access memory for real-time data processing and multitasking. It features a dedicated embedded neural processing unit (600 GOPS NPU & 3 TOPS/W power consumption) and an image signal processor (ISP) supporting parallel and MIPI CSI-2 camera module interfaces. It supports a 5 Mpixel camera at 30 FPS and includes features such as bad pixel correction, downsampling, black level adjustment, exposure control, demosaicing, column conversion, contrast adjustment, cropping, scaling, gamma correction, and pixel packing. VD66GY: • A 1.5M BSI sensor (1124x1364) configurable in both RGB and RGB-IR modes, with a dynamic range of up to 60dB and a pixel size of 2.61μm. It performs exceptionally well in both visible light and near-infrared regions (940nm). These sensors can capture up to 88 frames per second at a resolution of 1124x1364 and offer two pixel matrix versions: RGB 2x2 core and RGB-NIR 4x4 core. The Global Shutter CIS is ideal for machine vision, ensuring undistorted images during motion, which is beneficial for obtaining complete images for analysis.
►方案规格
STM32N6 Specifications: 1. Arm 32-bit Cortex-M55, 3360 CoreMark, up to 800 MHz frequency, 32 kbyte ICACHE, 32 kbyte DCACHE 2. Arm MVE (M-Profile Vector Extension), helium™ technology, TrustZone MPU, NVIC 3. Equipped with half-precision, single-precision, and double-precision floating-point units (FPU), supporting vector and scalar half-precision, single-precision, and double-precision floating-point data types 4. ST Neural-ART accelerator, up to 1 GHz frequency, 600 Gops, 288 MAC/cycle 5. Dedicated hardware unit for deep neural network (DNN) inference, flexible dedicated stream processing engine 6. 4.2 Mbyte SRAM 7. 128 KB tightly coupled TCM (synchronous memory) RAM with error correction for storing critical real-time data; 64 KB tightly coupled TCM RAM (also with error correction) for storing critical real-time program code 8. 128 kbyte TCM (tightly coupled memory) RAM with ECC for critical real-time data + 64 kbyte instruction TCM RAM with ECC for critical real-time routines 9. Arm TrustZone and secure I/O memory and peripherals, SESIP Level 3, Arm PSA (Platform Security Architecture) certification 10. Flexible lifecycle scheme, including RDP functionality and password-protected debugging features 11. Secure boot code in ROM for decrypting and authenticating customer-updatable "Root of Trust" (uRoT), secure data storage enabled by hardware unique key (HUK) VD66GY Specifications: 1. 1.5-megapixel sensor (1124x1364), available in RGB and RGB-NIR mode versions 2. 3D stacked sensor with BSI pixel size of 2.61 µm x 2.61 µm, image array size of 2.9 mm x 3.6 mm, optical format of 4 inches 3. Dimensions: 3.6 mm x 4.3 mm 4. Global shutter technology 5. Dual-channel transmitter MIPI CSI-2 (Copyright © 2005-2010 MIPI Alliance, Inc. Camera Serial Interface 2 (CSI-2) Standard Version 1.0) Version 1.3, with a transmission rate of 1.5 Gbps per channel 6. Fast mode plus CCI control interface 7. Linear dynamic range up to 60 dB 8. Integrated temperature sensor 9. Up to 88 frames per second in full resolution mode 10. 8 programmable GPIO ports for controlling LED outputs, synchronized with sensor integration cycles, pulse-width modulation control, master/slave external frame start, mirroring/flipping readout 11. Operating junction temperature: -30°C to 85°C