1. Overview
As large language models like ChatGPT and Gemini gradually integrate into our daily lives, the term 'AI' has become deeply ingrained in people's minds. Therefore, making people's work smarter, more efficient, and faster has become the top priority of the next era! This has also driven a surge in the demand for GPUs in cloud servers. According to a DIGITIMES report— Demand for AI servers is surging, and the market value of high-end server GPUs is expected to reach $102.2 billion in 2024, with NVIDIA leading in market share.』。
Additionally, another topic worth paying attention to is the twin brother of cloud applications.Edge ComputingThe application covers smartphones, cameras, robots, cars, drones, wearable devices, industrial and medical computers, enabling every device to have AI intelligence. Compared to cloud computing, which relies on the internet, edge computing emphasizes concepts such as processing speed, energy efficiency, low power consumption, and privacy.
For those who have a certain understanding of computer vision, machine learning, and deep learning, they will understand that these two fields can actually be divided into 'training' and 'inference.' Due to the hardware design of GPUs being highly suitable for tasks that are both highly repetitive and complex, using GPUs to train AI models is very appropriate! Once the model is trained, it can be used for extended periods to perform various prediction tasks, such as determining the location of objects, matching facial features, or predicting what might be said next in AI tasks.Therefore, the 'reasoning' application market is quite large and will continue to grow.Sentence: Sentence: . With Data Bridge – 『Global Edge Artificial Intelligence (AI) Hardware Market Size, Share, and Trends Analysis Report – Industry Overview and Forecast to 2032 The current market size of Edge AI hardware is $1.86 billion, and it is expected to reach $4.94 billion by 2032.
Edge Computing Market Forecast –Source: Data Bridge website
2. Drivers of the New Wave of AI
The rapid development of global artificial intelligence (AI) technology is driving the entire semiconductor industry into a new era, and among themInferenceWhat is neededComputing power (TOPS, Tera Operations Per Second, trillion operations per second)It has become one of the most direct and important indicators for measuring the performance of AI chips. This demand has not only reshaped the design philosophy of hardware but also initiated a global new battleground for AI chips!
Emerging AI chip companies are flourishing, driving innovation in NPU architecture.
Over the past five years, with the rise of the artificial intelligence inference market, an increasing number of startups have been investing in the development of dedicated AI accelerators, particularly targeting the demand for low-power, high-performance edge devices. This has sparked a wave of innovation in neural processing unit (NPU) architectures. Below are some of the internationally representative AI chip startups: starting from those established in 2015.Kneron (Taiwan)Established in 2017Hailo (Israel)Established in 2019 DeepX (South Korea)、 MemryX (United States)、 Axelera AI (Netherlands)withNeuchips (Taiwan)By 2025, there will already be more than30 startup companiesEntering this field, creating a rare phenomenon for this generation.A hundred schools of thought contendThe approach not only starts with innovations in hardware architecture but also extends to software ecosystem integration, model transformation, and toolchain design, forming a comprehensive AI acceleration ecosystem.
The swift counterattack of traditional semiconductor giants
Faced with the rise of emerging AI chip companies, traditional chip giants that once dominated the semiconductor market are also accelerating their pace, actively transforming to adapt to new trends in AI inference and edge computing. These companies, equipped with advanced manufacturing processes, comprehensive development tools, and global supply chain advantages, are leveraging mergers and acquisitions, product integration, and ecosystem development to stay competitive.Reshaping the AI hardware landscapeFor example, Qualcomm has integrated the AI engine into the Snapdragon platform, which is widely used in smartphones and in-vehicle devices; NXP integrates NPU through its i.MX series.eIQ software tools, targeting industrial control and smart cities; Renesas has introduced Reality AIDesigned for MCU/MPU platforms, it enhances sensing and real-time control applications. Meanwhile, ARM, as the world's largest IP core provider, has also entered the fray by launching the Ethos-U series NPUs, specifically designed for low-power devices and tightly integrated with Cortex-M / Cortex-A processors.Rapidly sweeping through the wearable, home, IoT, and other edge AI markets, as defined in the image below.。
ARM analysis of computational power application market diagramSource: ARM and linuxgizmos websites
The Era of AI Chip Diversification: Dual-Axis Evolution of Vision and Language Models
With the rapid growth of AI inference demands, the market has clearly divided intoTwo key application areas: Computer Vision and Large Language Models (LLM)Each AI chip manufacturer designs corresponding architectures and product strategies based on different needs.
AtComputer VisionThe domain covers application scenarios such as smart transportation, autonomous driving, security monitoring, and industrial image recognition, with a particular emphasis on real-time performance, low latency, and low power consumption. Therefore, most chip designs adopt mid-level computing power (10~50 TOPS) and optimize data migration and model deployment efficiency, for example:DEEPX DX-M1(25 TOPS)Hailo-8(26 TOPS)MemryX MX3(24 TOPS), etc. Additionally, for exampleSiMa.ai MLSoC(50 TOPS) andAxelera AI Metis(214 TOPS) high-performance products can also support the requirements for multi-model parallel processing or high-resolution visual analysis.
In comparison,Large language modelFor models like ChatGPT, BERT, and LLaMA, extremely high computational throughput and parameter processing capabilities are required. This has driven AI chips to evolve toward high TOPS, high bandwidth, and large internal storage. For example:Tenstorrent Wormhole n300dThe inference performance reaches 466 TOPS andNeuchips RecAccel N3000 (206 TOPS)Optimized for data centers and cloud deployments, products specifically designed for edge deployments. Smaller edge devices, such as...DEEPX DX-M2(40 TOPS)Hailo-10H(40 TOPS) is already capable of supporting lightweight LLM inference.
As shown in the figure below, it not only highlights the technical positioning of various chip products but also illustrates that the AI hardware market is rapidly moving toward a trend of application stratification and scenario-based design. Evaluating the strengths and weaknesses of AI chip suppliers is not solely based onComputing power (TOPS) as the sole measurement standardIt is essential to consider power consumption, latency, and the actual software deployment experience. Users will be able to find the most suitable NPU solution from this.This AI chip competition is not just about performance; it's a comprehensive contest of system integration and application implementation capabilities.。
AI chip market layout diagram (Information and logos sourced from public websites, for reference only)
The core key of edge AI: Precisely matching chips to specific scenarios.
A truly effective AI system in the field relies on much more than just a single NPU chip. While the NPU is the key computational core for neural network inference, it is merely one of many processing modules within the overall system. In practical applications, achieving smooth and efficient AI processing also requires the support of co-processors within the SoC, such as those responsible for image preprocessing.ISP (Image Signal Processor)Assisting with graphic rendering and compositionGPU (Graphics Processing Unit)And the VPU (Video Processing Unit), which is responsible for video decoding, encoding, and compression. These co-processors play an indispensable role, from image capture, frame correction, and format conversion to the pre-processing and post-processing of AI models—every step is crucial. Through the high-level collaboration of these modules, AI chips can achieve truly commercially viable application benefits.
As shown in the figure below, the practical implementation of AI chips should not be limited to a comparison of inference performance (TOPS). Instead, it should be approached with design considerations based on the overall system architecture. From the front-end camera to the mid-tier NVR (Network Video Recorder), and finally to the back-end server, each stage may involve different types, scales, and capabilities of AI processors.
WithSmart Front-End CameraFor example, such products require features like built-in image capture, real-time inference, low-latency feedback, and low power consumption, and therefore are more commonly designed using technologies such as Kneron KL720(1.5 TOPS) andNXP i.MX 8M Plus(2.3 TOPS) This highly integrated SoC chip equipped with multimedia modules is not only compact in size but also features basic AI inference and ISP support capabilities. It can directly perform tasks such as facial detection and object recognition at the edge. In mid-tier applications, such as NVRs, it is often unnecessary for every camera to have AI functionality, as centralized inference for multiple streams is typically handled by the NVR. These types of products can be paired with solutions such as... Hailo-8(26 TOPS) andDeepX DX-M1（25 TOPS) and other mid-to-high performance AI modules, supporting simultaneous video analysis for multiple or even dozens of cameras, are currently common solutions in smart city and retail scenarios.
As for the backend edge servers and data centers, they need to handle larger-scale computational loads and more complex models, especially with applications like modern voice assistants, real-time translation, and large language models (LLMs). At this point, it becomes necessary to equip them with technologies such as Neuchips RecAccel(206 TOPS), even Tenstorrent Wormhole n300d(466 TOPS) high-end AI chips with powerful inference performance, memory bandwidth, and multitasking capabilities can support large-scale real-time inference, vision-language integration, or hybrid cloud-edge deployments. Overall, AI chips are not a competition of single components.Instead, it is a competition in system design that tests 'integration capabilities.'Only by organically integrating the NPU with modules such as the ISP, GPU, and VPU, and precisely deploying AI hardware based on application scenarios, can the true commercial value of AI systems be realized.
AI product design schematic
3. Conclusion
Looking at the development trajectory of the AI chip industry, it has evolved fromThe early focus on a single metric, such as computational performance (e.g., TOPS), is gradually shifting toward a new competitive landscape centered on application-oriented approaches, system integration, and real-world implementation.Whether deployed in smart cameras, industrial equipment, voice assistants, or large language model servers, the value of an AI chip is no longer just about speed and power in numbers. It lies in its ability to truly integrate into products, create synergistic effects, and solve real-world problems.
Although the NPU is the core of AI computing, it can only create truly commercially valuable smart terminals when it is highly integrated with co-processors such as ISP, GPU, and VPU, along with efficient system design and software support. However,An excellent AI chip is never just about the hardware itself; it also relies heavily on the construction of a comprehensive ecosystem.From driver support, model conversion tools, and development frameworks, to the continuous optimization of sample programs and deployment templates, these are all critical factors in attracting developers and integrating products. Additionally, close communication with users and feedback loops are indispensable for accelerating product maturity and market expansion.
To truly lead the market and challenge the dominance of established giants like NVIDIA, AI chip suppliers must possess robust system integration capabilities, flexible deployment solutions, and a developer-centric software support strategy. Only then can they secure their footing in this AI revolution and carve out new competitive advantages.
Therefore,The AI community of DadaTong For users evaluating AI chips, we provide the most extensive resource support, including official demo videos, developer resources, example applications (Example Code), and module resources (Model Zoo).Assist edge computing partners in quickly identifying opportunities for AI application implementation!
4. Reference Documents
Reference website:
[2] Global Edge Artificial Intelligence (AI) Hardware Market Size, Share, and Trend Analysis Report – Industry Overview and 2032 Forecast
[3]Arm has released two lightweight edge AI neural processing units (NPUs).
[5] The eIQ® toolkit is used for end-to-end model development and deployment.
[6]Renesas acquires Reality AI and announces advancements in AI and TinyML solutions.
If there is anything related.Artificial IntelligenceFor technical issues, feel free to leave a comment below the article!
More will be shared next.Artificial IntelligenceTechnical articles !!Stay tuned for 【ATU Book - AI Series】 !!