In this tutorial, we work through an end-to-end workflow for Qualcomm AI Hub Models. We start by setting up the required package, discovering the available model collection, and loading MobileNet-V2 for local PyTorch inference. We also handle an important input-shape issue by converting NHWC image tensors into the NCHW format expected by the model. From there, we run inference on both the model’s built-in sample input and a real image, inspect top predictions, execute the official Qualcomm AI Hub CLI demo, and extend the workflow with a YOLOv7 object detection example. Also, we include an optional cloud-device section where we compile, profile, and run the model on a real Qualcomm device when an API token is available.
import subprocess, sys, os, glob, textwrap, traceback
import numpy as np, torch
from PIL import Image
import matplotlib.pyplot as plt
def pip_install(*pkgs):
subprocess.run([sys.executable, "-m", "pip", "install", "-q", *pkgs], check=True)
pip_install("qai_hub_models")
OUT_DIR = "/content/qaihm_out"; os.makedirs(OUT_DIR, exist_ok=True)
torch.set_grad_enabled(False)
def to_nchw(value):
arr = value[0] if isinstance(value, (list, tuple)) else value
t = torch.from_numpy(np.asarray(arr, dtype=np.float32))
if t.ndim == 3:
t = t.unsqueeze(0)
if t.ndim == 4 and t.shape[1] != 3 and t.shape[-1] == 3:
t = t.permute(0, 3, 1, 2).contiguous()
return tWe begin by importing libraries and setting up a helper function to install packages directly inside Colab. We install qai_hub_models, create an output directory, and disable gradient tracking since we only need inference. We also define the to_nchw() function to convert any input image tensor to the channel-first format expected by the model.
import pkgutil, qai_hub_models.models as _m
model_ids = sorted(n for _, n, p in pkgutil.iter_modules(_m.__path__)
if p and not n.startswith("_"))
print(f">>> {len(model_ids)} models available. First 40:\n")
print(textwrap.fill(", ".join(model_ids[:40]), 100), "\n")
from qai_hub_models.models.mobilenet_v2 import Model as MobileNetV2
model = MobileNetV2.from_pretrained().eval()
spec = model.get_input_spec()
input_name = list(spec.keys())[0]
print(">>> Input:", input_name, spec[input_name].shape, spec[input_name].dtype)
from torchvision.models import MobileNet_V2_Weights
IMAGENET_CLASSES = MobileNet_V2_Weights.IMAGENET1K_V1.meta["categories"]
def top5(logits):
if logits.ndim == 1: logits = logits.unsqueeze(0)
probs = torch.softmax(logits, dim=1)[0]
conf, idx = probs.topk(5)
return [(IMAGENET_CLASSES[i], float(c)) for c, i in zip(conf, idx)]We discover the available Qualcomm AI Hub model packages and print the first set of model IDs to understand what is accessible. We then load the pretrained MobileNet-V2 model, read its input specification, and identify the correct input name. We also prepare the ImageNet class labels and define a top5() function to convert model logits into readable top-5 predictions.
sample = model.sample_inputs()
x = to_nchw(sample[input_name])
print(">>> fed tensor shape:", tuple(x.shape))
print("\n>>> Top-5 for the built-in sample input:")
for label, conf in top5(model(x)):
print(f" {conf:6.2%} {label}")
from torchvision import transforms
preprocess = transforms.Compose([
transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(),
])
img = None
try:
import urllib.request
p = os.path.join(OUT_DIR, "input.jpg")
urllib.request.urlretrieve(
"https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg", p)
img = Image.open(p).convert("RGB")
except Exception as e:
print(">>> photo download skipped:", e)
if img is not None:
preds = top5(model(preprocess(img).unsqueeze(0)))
print("\n>>> Top-5 for the downloaded photo:")
for label, conf in preds: print(f" {conf:6.2%} {label}")
plt.figure(figsize=(5,5)); plt.imshow(img); plt.axis("off")
plt.title(f"{preds[0][0]} ({preds[0][1]:.1%})"); plt.show()We first run inference using the model’s built-in sample input and use to_nchw() to fix the tensor shape before passing it to MobileNet-V2. We then download a real image, preprocess it using standard resizing, cropping, and tensor conversion steps, and run another prediction. We finally display the image with the top predicted label to visually connect the model output to the input photo.
def run_demo(module, extra=None, timeout=900):
cmd = [sys.executable, "-m", module, "--eval-mode", "fp",
"--output-dir", OUT_DIR] + (extra or [])
print(f"\n>>> {' '.join(cmd)}")
try:
r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
print("\n".join((r.stdout + r.stderr).strip().splitlines()[-25:]))
except Exception as e:
print(">>> demo skipped:", e)
run_demo("qai_hub_models.models.mobilenet_v2.demo")
try:
pip_install("qai_hub_models[yolov7]")
run_demo("qai_hub_models.models.yolov7.demo")
imgs = sorted(glob.glob(OUT_DIR + "/*.png") + glob.glob(OUT_DIR + "/*.jpg"),
key=os.path.getmtime)
if imgs:
plt.figure(figsize=(9,9)); plt.imshow(Image.open(imgs[-1]).convert("RGB"))
plt.axis("off"); plt.title("YOLOv7 detections"); plt.show()
else:
print(">>> no output image found (results may have printed instead).")
except Exception:
print(">>> YOLOv7 section skipped:\n", traceback.format_exc())We define a reusable run_demo() function that executes official Qualcomm AI Hub model demos from the command line. We use it to run the MobileNet-V2 demo and then install the YOLOv7 extras for object detection. We run the YOLOv7 demo, search for the generated output image, and visualize the detections if an image is created.
try:
import qai_hub as hub
devices = hub.get_devices()
print(f"\n>>> Authenticated. {len(devices)} cloud devices available.")
device = hub.Device("Samsung Galaxy S24 (Family)")
sample = model.sample_inputs()
nchw = to_nchw(sample[input_name])
traced = torch.jit.trace(model, [nchw])
cloud_inputs = {input_name: [nchw.numpy()]}
cj = hub.submit_compile_job(model=traced, device=device,
input_specs=model.get_input_spec(),
options="--target_runtime tflite")
target = cj.get_target_model(); print(">>> compiled:", cj.url)
pj = hub.submit_profile_job(model=target, device=device); print(">>> profiling:", pj.url)
ij = hub.submit_inference_job(model=target, device=device, inputs=cloud_inputs)
out = ij.download_output_data()
dev_logits = torch.from_numpy(np.asarray(list(out.values())[0][0]))
print(">>> Top-5 from the REAL device:")
for label, conf in top5(dev_logits): print(f" {conf:6.2%} {label}")
target.download(os.path.join(OUT_DIR, "mobilenet_v2.tflite"))
print(">>> saved compiled .tflite to", OUT_DIR)
except Exception as e:
print("\n>>> Cloud (on-device) section skipped — no API token configured.")
print(" Get one at workbench.aihub.qualcomm.com, then:")
print(" !qai-hub configure --api_token YOUR_TOKEN")
print(" detail:", (str(e).splitlines() or [type(e).__name__])[0])
print("\n>>> Tutorial complete. Outputs in:", OUT_DIR)We include an optional Qualcomm AI Hub cloud workflow that runs only when an API token is configured. We retrieve available cloud devices, trace the PyTorch model, compile it for TFLite, profile it on a Qualcomm device, and submit an inference job. We then download the device output, print the top predictions, save the compiled TFLite model, and finish by showing where all tutorial outputs are stored.
In conclusion, we have a complete practical workflow for using Qualcomm AI Hub Models inside Colab. We learned how to load pretrained models, prepare inputs correctly, run local inference, visualize classification and detection results, and use the official demos as reproducible reference points. We also saw how the same model can move beyond local PyTorch execution into Qualcomm’s cloud-device pipeline for compilation, profiling, and real-device inference. It provides a path from simple experimentation to hardware-aware deployment with Qualcomm AI Hub.
Check out the Full Codes with Notebook here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post A Hands-On Coding Tutorial on Qualcomm AI Hub Models for Classification, Object Detection, and Hardware-Aware Deployment appeared first on MarkTechPost.
Credit: Source link
