MediaPipe for Raspberry Pi and iOS– Google for Developers Blog Site

Published by Paul Ruiz, Designer Relations Engineer

Back in Might we launched MediaPipe Solutions, a set of tools for no-code and low-code services to typical on-device maker finding out jobs, for Android, web, and Python. Today we more than happy to reveal that the preliminary variation of the iOS SDK, plus an upgrade for the Python SDK to support the Raspberry Pi, are offered. These consist of assistance for audio category, face landmark detection, and numerous natural language processing jobs. Let’s have a look at how you can utilize these tools for the brand-new platforms.

Item Detection for Raspberry Pi

Aside from establishing your Raspberry Pi hardware with an electronic camera, you can begin by setting up the MediaPipe reliance, in addition to OpenCV and NumPy if you do not have them currently.

 python -m pip set up mediapipe

From there you can develop a brand-new Python file and include your imports to the top.

 import mediapipe  as mp

 from mediapipe. jobs  import python

 from mediapipe. jobs python  import vision

 import cv2

 import numpy  as np

You will likewise wish to make certain you have an item detection design kept in your area on your Raspberry Pi. For your benefit, we have actually supplied a default design, EfficientDet-Lite0, that you can recover with the following command.

wget -q -O efficientdet.tflite -q https://storage.googleapis.com/mediapipe-models/object_detector/efficientdet_lite0/int8/1/efficientdet_lite0.tflite

When you have your design downloaded, you can begin producing your brand-new ObjectDetector, consisting of some personalizations, like limit results that you wish to get, or the self-confidence limit that need to be surpassed prior to an outcome can be returned.



 base_options = python.BaseOptions( model_asset_path= design)

alternatives = vision.ObjectDetectorOptions( base_options = base_options, running_mode= vision.RunningMode.LIVE _ STREAM, max_results= max_results, score_threshold= score_threshold,

           result_callback= save_result).

detector = vision.ObjectDetector.create _ from_options( alternatives)

After producing the ObjectDetector, you will require to open the Raspberry Pi video camera to check out the constant frames. There are a couple of preprocessing actions that will be left out here, however are offered in our sample on GitHub.

Within that loop you can transform the processed video camera image into a brand-new MediaPipe.Image, then run detection on that brand-new MediaPipe.Image prior to showing the outcomes that are gotten in an associated listener.

 mp_image = mp.Image( image_format= mp.ImageFormat.SRGB, information= rgb_image).

detector.detect _ async( mp_image, time.time _ ns())

When you extract those outcomes and discovered bounding boxes, you must have the ability to see something like this:

Moving image of a person holidng up a cup and a phone, and detected bounded boxes identifying these items in real time

You can discover the total Raspberry Pi example revealed above on GitHub, or see the main paperwork here

Text Category on iOS

While text category is among the more direct examples, the core concepts will still use to the remainder of the offered iOS Tasks. Comparable to the Raspberry Pi, you’ll begin by producing a brand-new MediaPipe Tasks item, which in this case is a TextClassifier

 var textClassifier: TextClassifier?

textClassifier = TextClassifier( modelPath: model.modelPath)

Now that you have your TextClassifier, you simply require to pass a String to it to get a TextClassifierResult

 func categorize( text: String) -> > TextClassifierResult? {

guard let textClassifier = textClassifier  else {

 return nil.

}

return attempt? textClassifier.classify( text: text). }

You can do this from in other places in your app, such as a ViewController DispatchQueue, prior to showing the outcomes.

 let outcome = self?. textClassifier.classify( text: inputText).

let classifications = outcome?. classificationResult.classifications.first?. classifications ?? []

You can discover the remainder of the code for this job on GitHub, in addition to see the complete paperwork on developers.google.com/mediapipe