NVIDIA Metropolis Microservices for Jetson

Announcing NVIDIA Metropolis Microservices for Jetson for Rapid Edge AI Development

Building vision AI applications for the edge often comes with notoriously long and costly development cycles. At the same time, quickly developing edge AI applications that are cloud-native, flexible, and secure has never been more important. Now, a powerful yet simple API-driven edge AI development workflow is available with the new NVIDIA Metropolis microservices.

NVIDIA Metropolis microservices is a suite of customizable, cloud-native building blocks for developing vision AI applications and solutions. This release introduces an expanded set of APIs and microservices on the NVIDIA Jetson platform to further accelerate the development and deployment of vision AI applications at the edge.

These new Jetson microservices empower developers to modernize their AI application stack, streamline processes, and safeguard applications for the future. You can easily incorporate the latest in generative AI advancements through APIs and microservices such as video storage and management, prebuilt AI perception pipelines, tracking algorithms, system monitoring, IoT services for secure edge-to-cloud connectivity, and more.

A graphic that compares using a do-it-yourself approach, where developers have to create all the pieces from scratch, and using prebuilt NVIDIA Metropolis microservices, which reduce development time from years to months.

Figure 1. Develop edge AI applications faster with NVIDIA Metropolis microservices

Release highlights

• Production-ready edge AI applications require numerous components, including AI models, optimized processing and inference pipelines, glue logic, security measures, cloud connectivity, and more. NVIDIA Metropolis microservices for Jetson streamline the application development process by offering pre-built microservices for the most ubiquitous components using a cloud-native, modular, and extensible architecture.
• The flexibility of the platform enhances development efficiency, with standard APIs seamlessly integrating with other applications and services. The platform also provides essential services such as IoT, security, and monitoring, offering ready-to-use core components for production applications.
• With access to more than 15 microservices across Application, Platform services, and Cloud services, developers are freed to concentrate on building intellectual property and achieving differentiation in the market.

NVIDIA partners are incorporating NVIDIA Metropolis microservices into their offerings, including AAEON, Aetina, Advantech, Allxon, CRG, CVEDIA, Namla, Rebotnix, RidgeRun, Seeed Studio, and Silicon Highway. More are added daily.

Reference workflows and applications

Two reference applications are included to help get you started with NVIDIA Metropolis microservices for Jetson: the AI-enabled network video recorder (AI-NVR) and a generative AI application with zero-shot detection. These workflows show how the microservices and APIs come together to build complete applications from video ingestion, AI inference, analytics, and monitoring to securely connecting to the cloud.

AI-enabled network video recorder

The AI-NVR reference workflow brings nearly all the microservices together in one comprehensive app. It includes:

• Video ingestion and storage using the Video Storage Toolkit (VST) microservice
• People detection and tracking with the AI Perception service with NVIDIA DeepStream
• Line crossing and Region of Interest (ROI) insights and alerts using the Analytics service
• An Android reference mobile application is provided to demonstrate the use of APIs to build client applications. To learn more, check out the NVIDIA On-Demand playlist, AI-NVR Using Metropolis Microservices for Jetson.

Graphic showing the complete cloud-native architecture of AI-NVR application showing VST, AI perception service, analytics service, and all the other platform services.

Figure 2. The AI-enabled network video recorder (AI-NVR) application architecture

Zero-shot detection using generative AI

Metropolis microservices for Jetson enables developers to prototype and productize generative AI applications for the edge. The generative AI reference application enables zero-shot detection of live streaming data. Models can detect any objects specified with a prompt.

Prompts can be made remotely over REST APIs to the AI Perception service to dynamically change classes to detect. Generative AI enables a new breed of AI-powered applications for the edge. To learn more about generative AI with Metropolis microservices, see Bringing Generative AI to the Edge with NVIDIA Metropolis Microservices for Jetson.

Graphic showing the cloud-native architecture of the generative AI reference application for zero-shot object detection using a visual prompting agent.

Figure 3. Generative AI reference application for zero-shot object detection

Powerful microservices and APIs

Metropolis microservices for Jetson is a collection of feature-rich microservices and APIs, including application services, platform and Board Support Package (BSP) services, and cloud services. The modular and extensible microservices make it easy to build modern cloud-native applications for the edge.

As a developer, you have the flexibility to choose one, several, or all of the services depending on the maturity of your product.

Graphic showing the complete software stack from the reference AI workflow, application microservice, platform, and BSP services to cloud services.

Figure 4. Metropolis microservices for Jetson software stack

Application services

• Video Storage Toolkit: Service for video ingestion and storage
• AI Perception service using NVIDIA DeepStream: For AI inference, object tracking and metadata creation
• AI Perception service for zero-shot detection: For generative AI inference with the NanoOWL model and visual prompting
• Analytics service: Object counting analytics such as line crossing, Region of Interest, and Field of View

Platform services

• Redis: Global message bus for inter-process communication
• API Gateway: Maps incoming API requests to the subsequent services
• Monitoring: Monitor and visualize edge device status such as utilization and app KPIs
• IoT Gateway: Secure bidirectional communication between edge and cloud

Cloud services

IoT Cloud: Create secure connection from cloud to edge, including authentication and authorization

By Chintan Shah and Debraj Sinha/NVDIA