Processing IP selection: workload first, headline TOPS second

Define the workload before comparing engines

VPU / Processing spans different workloads: video codecs, vector processors and neural-network accelerators are not interchangeable. Identify algorithm, input dimensions, precision, output quality and deadlines. We use neural-network compilation as one concrete example, not as a definition of every VPU.

Prepare normal and difficult inputs, large tensors and concurrent SoC activity. Freeze model or algorithm versions and quality criteria. A benchmark that silently changes precision or output quality is not a like-for-like comparison.

Check the memory-bound ceiling

Berkeley Lab’s Roofline model connects compute throughput, memory bandwidth and arithmetic intensity. Peak compute alone cannot describe attainable performance. The following is our illustrative integer-operation adaptation, not a measurement or vendor claim.

Assume 4 TOPS peak compute, 16 GB/s sustainable bandwidth at the chosen memory boundary and 100 operations per byte crossing that boundary. The bandwidth ceiling is 16 × 10⁹ × 100 = 1.6 TOPS. The simple bound is min(4, 1.6) = 1.6 TOPS; actual performance may be lower. Keep precision and operation counting—including the definition of a MAC—consistent.

Sources: 1

Inspect what actually runs on the accelerator

Arm’s Vela documentation gives a concrete example: unsupported regions in TFLite models remain CPU operations, whereas its TOSA compilation path has no CPU fallback. Vela reports placement and estimates for cycles, bandwidth and memory. This describes that toolchain, not all processing IP.

Request compilation reports for your workload. Separate accelerator execution, CPU fallback, transfers, synchronization and preprocessing. Check peak SRAM use and external traffic, not only model size. Record compiler version, settings and memory configuration to make results reproducible.

Sources: 2

Accept end-to-end behavior

Measure latency distributions at the required batch size, sustained task rate, accuracy and energy within a defined system boundary. Include thermal steady state and contention where relevant. Clearly distinguish simulation estimates from silicon measurements and record clocks and voltage conditions.

Finally review supported operators or instructions, runtime ownership, licensing, debugger access and the update path when workloads change. Select the configuration that meets the full workload contract with reproducible evidence and maintainable software, rather than the largest headline TOPS figure.

Bring these to your IP evaluation

• Provide representative workloads and quality targets.
• Review placement, SRAM demand and memory traffic.
• Compare end-to-end latency, energy and software support.