The ML embedded evaluation kit provides a range of ready to use machine learning (ML) applications for users to develop ML workloads running on the Arm® Ethos-U NPU and Arm® Cortex-M CPUs. You can also access metrics such as inference cycle count to estimate performance.
The Arm® Ethos-U NPU is a new class of ML processor, specifically designed to accelerate ML computation in constrained embedded and IoT devices.
Experiment with the included end-to-end software use cases and create your own ML applications for Cortex-M CPU and Ethos-U NPU.
| ML application | Description | Neural Network Model |
|---|---|---|
| Image classification | Recognize the presence of objects in a given image | Mobilenet V2 |
| Keyword spotting(KWS) | Recognize the presence of a key word in a recording | MicroNet |
| Automated Speech Recognition(ASR) | Transcribe words in a recording | Wav2Letter |
| KWS and ASR | Utilise Cortex-M and Ethos-U to transcribe words in a recording after a keyword was spotted | MicroNet Wav2Letter |
| Anomaly Detection | Detecting abnormal behavior based on a sound recording of a machine | MicroNet |
| Visual Wake Word | Recognize if person is present in a given image | MicroNet |
| Noise Reduction | Remove noise from audio while keeping speech intact | RNNoise |
| Object detection | Detects and draws face bounding box in a given image | Yolo Fastest |
| Generic inference runner | Code block allowing you to develop your own use case for Ethos-U NPU | Your custom model |
This repository is for building and deploying Machine Learning (ML) applications targeted for Arm® Cortex®-M and Arm® Ethos™-U NPU. To run evaluations using this software, we suggest using:
-
MPS3 board with Arm® Corstone-300 or Arm® Corstone-310 implementations.
- Arm® Corstone™-300 runs a combination of the Arm® Cortex™-M55 processor and the Arm® Ethos™-U55 NPU.
- Arm® Corstone™-310 runs a combination of the Arm® Cortex™-M85 processor and the Arm® Ethos™-U55 NPU.
-
Arm® Corstone™ Fixed Virtual Platform (FVP)
- Arm® Corstone™-300 MPS3 based Fixed Virtual Platform (FVP) offers a choice of the Arm® Ethos™-U55 NPU or Arm® Ethos™-U65 NPU software fast model in combination with the Arm® Cortex™-M55 processor
- Arm® Corstone™-310 MPS3 based Fixed Virtual Platform (FVP) offers a choice of the Arm® Ethos™-U55 NPU or Arm® Ethos™-U65 NPU software fast model in combination with the Arm® Cortex™-M55 processor
- Arm® Corstone™-315 MPS3 based Fixed Virtual Platform (FVP) offers the Arm® Ethos™-U65 NPU software fast model in combination with the Arm® Cortex™-M85 processor
- Arm® Corstone™-320 MPS3 based Fixed Virtual Platform (FVP) offers the Arm® Ethos™-U85 NPU software fast model in combination with the Arm® Cortex™-M85 processor
- You can also take advantage of Arm Virtual Hardware (AVH) and run the Fixed Virtual Platform in the cloud.
Arm® Corstone™-300 and Corstone™-310 design implementations are publicly available on Download FPGA Images page, or as a Fixed Virtual Platform of the MPS3 development board.
Arm® Corstone™-315 and Corstone™-320 design implementations are publicly available as a Fixed Virtual Platform of the MPS4 development board.
To run ML applications on the Cortex-M and Ethos-U NPU:
-
First, verify that you have installed all of the required prerequisites.
NOTE:
Dockerfileis also available if you would like to build a Docker image. -
Clone the Ethos-U evaluation kit repository:
git clone "https://git.gitlab.arm.com/artificial-intelligence/ethos-u/ml-embedded-evaluation-kit.git" cd ml-embedded-evaluation-kit -
Pull all the external dependencies with the following command:
git submodule update --init -
Next, run the
build_defaultPython script. It handles the downloading of the neural network models, compiling using Vela, and building the project using CMake.
python3.10 ./build_default.py --toolchain arm
python3.10 ./build_default.py
-
Change directory to the generated cmake build folder which contains the
.axffile output in thebinsubdirectory. Launch the application by passing the.axfto the FVP you downloaded when installing the prerequisites. Alternatively, from the root directory add<cmake-build-your_config>to the path to the axf and use one of the following commands:From auto-generated (or custom) build directory: <path_to_FVP>/FVP_Corstone_SSE-300_Ethos-U55 -a ./bin/ethos-u-kws.axf From root directory: <path_to_FVP>/FVP_Corstone_SSE-300_Ethos-U55 -a <cmake-build-your_config>/bin/ethos-u-kws.axf -
A telnet window is launched through which you can interact with the application and obtain performance figures.
For more details, you can view the quick start guide.
Note: The default flow assumes Arm® Ethos™-U55 NPU usage, configured to use 128 Multiply-Accumulate units and is sharing SRAM with the Arm® Cortex®-M55.
Ml embedded evaluation kit supports:
Ethos™-U NPU Default MACs/cc Other MACs/cc supported Default Memory Mode Other Memory Modes supported Ethos™-U55 128 32, 64, 256 Shared_Sram Sram_Only Ethos™-U65 256 512 Dedicated_Sram Sram_Only, Shared_Sram Ethos™-U85 256 128, 512, 1024, 2048 Dedicated_Sram Sram_Only, Shared_Sram For more information see Building.
See full documentation:
-
The ML use cases have common code such as initializing the Hardware Abstraction Layer (HAL)
-
The common application code can be run on native host machine (x86_64 or aarch64) or Arm Cortex-M architecture because of the HAL
-
Google® TensorFlow™ Lite for Microcontrollers inference engine is used to schedule the execution of neural network models
-
The Ethos-U NPU driver is integrated TensorFlow Lite for Microcontrollers
- ML operators are delegated to the NPU with CPU fall-back for unsupported operators
- CMSIS-NN is used to optimise CPU workload execution with int8 data type
- Final ML operator fall-back is TensorFlow™ Lite for Microcontrollers' reference kernels
-
The provided set of common ML use-case functions will assist in implementing your application logic
- When modifying use-case code, there is no requirement to modify other components of the eval kit
-
The CMake build system will discover and automatically include new ML application code into the compilation workflow
A high level overview of the different components in the software, and the platforms supported out-of-the-box, is shown in the diagram below.
Note: The Ethos-U NPU software stack is described here.
For a more detailed description of the build graph with all major components, see Building.
There are source files in the repository that form the core of the Machine Leaning flow for all the use cases. These are exposed as APIs that the examples can use and even be combined to form chained use cases. The API sources are designed to be portable across platforms and provide functionality for preprocessing of data, running an inference, and postprocessing of results. These allow a common flow for all use cases with minor differences in how each of these blocks are instantiated.
As an independent CMake project, these APIs can be used by or integrated into other projects easily. We also produce CMSIS Packs with these sources, so they could be used in all tools/IDEs (for example, Arm® Development Studio and Keil® µVision®) that support the use of CMSIS Packs.
The ML embedded eval kit welcomes contributions. For more details on contributing to the eval kit please see the the contributors guide.
Please, if you want to start public discussion, raise any issues or questions related to this repository, please create a new GitLab issue here: https://gitlab.arm.com/artificial-intelligence/ethos-u/ml-embedded-evaluation-kit/-/issues.
This product conforms to Arm's inclusive language policy and, to the best of our knowledge, does not contain any non-inclusive language. If you find something that concerns you, email [email protected].
The ML Embedded applications samples are provided under the Apache 2.0 license, see License Apache 2.0.
Application input data sample files are provided under their original license: