Edge and On-Device AI with ExecuTorch
PyTorch models on phones, wearables and embedded devices
Edge and On-Device AI with ExecuTorch
Some models belong on the device, not in the cloud. If your application needs sub-50 ms responses, has to work offline, processes sensitive data that should never leave the phone, or runs at a scale where per-inference cloud costs dominate, on-device AI is the answer. IntelliSensei deploys PyTorch models to mobile and embedded hardware with ExecuTorch, the PyTorch Foundation's official edge runtime.
Why on-device
Three reasons keep coming up with our clients. Latency: no network round trip, so interactive features feel instant. Privacy: camera frames, audio and health data are processed locally, which simplifies compliance and builds user trust. Cost: inference on hardware the user already owns scales to millions of users without a GPU bill. The trade-off is a hard compute and memory budget, and that is where the engineering lives.
ExecuTorch export pipeline
ExecuTorch is the PyTorch 2.x edge story: a model is captured with torch.export, lowered through a delegate backend for the target hardware, and run by a small C++ runtime with no Python on the device. We build and maintain that pipeline for you, targeting the delegates that match your fleet: Core ML and MPS for Apple devices, XNNPACK for portable CPU execution, Vulkan for mobile GPUs, and Qualcomm, MediaTek or Arm delegates for specific silicon. The result is a .pte artifact you version and ship like any other asset.
Model compression for the edge
Most models need to shrink before they fit. We apply quantization (INT8 and 4-bit where the delegate supports it), structured pruning and knowledge distillation, measuring accuracy after each step so you know exactly what the compression cost. For language models on device we work with the ExecuTorch LLM runners and quantized small models (1-3B parameters) that run at usable token rates on current phones.
Mobile and embedded integration
A model file is not a feature. We integrate the ExecuTorch runtime into your iOS (Swift) and Android (Kotlin) apps, or into embedded Linux and RTOS targets in C++, handle pre- and post-processing at native speed, and manage threading so inference never blocks the UI. We also benchmark on the real device matrix you ship to, not just the flagship in the office.
Fleet update and telemetry strategy
Models on devices need a lifecycle: how a new model version reaches users, how you roll back, how you know it is performing. We design over-the-air model delivery with staged rollouts, on-device A/B evaluation, and privacy-preserving telemetry (aggregate accuracy and latency, never raw inputs) so the model in the field keeps improving.
When the cloud is still right
We will tell you when on-device is the wrong call: models too large to compress usefully, workloads that need fresh server-side data, or a device matrix too fragmented to support. In those cases our cloud deployment and inference optimization services apply, and a hybrid design (small on-device model with a cloud fallback) is often the best of both.
Deliverables
An export pipeline in your CI, the lowered model artifacts for each target, native integration code, a device benchmark report, and a model-update mechanism.
Have a model that needs to run on a phone, a wearable or a board? Contact us and we will start with a feasibility benchmark on your target hardware.