# ParetoPilot decision summary

**Measured Arm64 result.** The required runner, source, model, and workload details are present. See the limits below.

Displayed measurements are rounded for readability; exact values remain in the machine-readable decision details.

## Decision

| Field | Value |
| --- | --- |
| Baseline | Q8 generic reference (q8\-generic) |
| Selected | Q8 generic reference (q8\-generic) |
| Baseline retained | Yes |
| Best objective result | Yes |
| Decision reason | The best measured result also ranked first under the stated preference order\. |

## How the cutoff was applied

| Field | Value |
| --- | --- |
| Objective | End-to-end latency p95 (ms) |
| Direction | Minimize |
| Best measured value | Q8 generic reference (q8\-generic) at 2,231.9329 |
| Tolerance defined before the run | 1% |
| Tolerance margin | 22.3193 |
| Shortlist boundary | 2,254.2522 (at\-or\-below) |
| Selected value | 2,231.9329 |
| Margin to shortlist boundary | 22.3193 (0.99% of boundary) |
| Closest candidate outside shortlist | Q4 with KleidiAI (q4\-kleidiai); value 2,299.4543; shortfall 45.2021 (2.01%) |

## Configuration comparison

Each stage changes one configuration setting. Deltas compare it with the stage immediately before it.

### Stage 1 — Reference

| Field | Value |
| --- | --- |
| Candidate | Q8 generic reference (q8\-generic) |
| What changed | reference |
| Objective (End-to-end latency p95 (ms)) | 2,231.9329 |
| Status | Baseline, Selected, Eligible, Pareto frontier, Objective shortlist |
| Constraint violations | None |

Starting point; no adjacent-stage delta applies.

### Stage 2 — Quantization

| Field | Value |
| --- | --- |
| Candidate | Q4 generic quantization (q4\-generic) |
| What changed | quantization |
| Objective (End-to-end latency p95 (ms)) | 2,311.1251 |
| Status | Eligible, Pareto frontier |
| Constraint violations | None |

Adjacent delta from Q8 generic reference (q8\-generic):

| Metric | Direction | Previous | Current | Delta | Delta percent | Effect |
| --- | --- | ---: | ---: | ---: | ---: | --- |
| End-to-end latency p95 (ms) | Minimize | 2,231.9329 | 2,311.1251 | +79.1923 | +3.55% | Tradeoff |
| Generation throughput (tokens/s) | Maximize | 38.7264 | 35.0123 | -3.7141 | -9.59% | Tradeoff |
| Model size (MiB) | Minimize | 1,806.7666 | 1,016.8335 | -789.9331 | -43.72% | Improved |
| Peak RSS (MiB) | Minimize | 3,437.5977 | 1,966.4727 | -1,471.125 | -42.8% | Improved |
| Prompt throughput (tokens/s) | Maximize | 102.6185 | 113.821 | +11.2025 | +10.92% | Improved |
| Quality score | Maximize | 0.875 | 0.8333 | -0.0417 | -4.76% | Tradeoff |
| Time to first token p95 (ms) | Minimize | 545.3739 | 483.1131 | -62.2608 | -11.42% | Improved |

**Not comparable between these stages:** None

### Stage 3 — KleidiAI build

| Field | Value |
| --- | --- |
| Candidate | Q4 with KleidiAI (q4\-kleidiai) |
| What changed | arm\-kernel |
| Objective (End-to-end latency p95 (ms)) | 2,299.4543 |
| Status | Eligible, Pareto frontier |
| Constraint violations | None |

Adjacent delta from Q4 generic quantization (q4\-generic):

| Metric | Direction | Previous | Current | Delta | Delta percent | Effect |
| --- | --- | ---: | ---: | ---: | ---: | --- |
| End-to-end latency p95 (ms) | Minimize | 2,311.1251 | 2,299.4543 | -11.6708 | -0.5% | Improved |
| Generation throughput (tokens/s) | Maximize | 35.0123 | 35.3764 | +0.364 | +1.04% | Improved |
| Model size (MiB) | Minimize | 1,016.8335 | 1,016.8335 | 0 | 0% | Held |
| Peak RSS (MiB) | Minimize | 1,966.4727 | 1,966.4844 | +0.0117 | +0.0005959% | Tradeoff |
| Prompt throughput (tokens/s) | Maximize | 113.821 | 114.448 | +0.627 | +0.55% | Improved |
| Quality score | Maximize | 0.8333 | 0.8333 | 0 | 0% | Held |
| Time to first token p95 (ms) | Minimize | 483.1131 | 470.4023 | -12.7109 | -2.63% | Improved |

**Not comparable between these stages:** None

### Stage 4 — Runtime tuning

| Field | Value |
| --- | --- |
| Candidate | Q4 with KleidiAI and 512\-token micro\-batch (q4\-kleidiai\-tuned) |
| What changed | runtime\-tuning |
| Objective (End-to-end latency p95 (ms)) | 2,307.7153 |
| Status | Eligible, Pareto frontier |
| Constraint violations | None |

Adjacent delta from Q4 with KleidiAI (q4\-kleidiai):

| Metric | Direction | Previous | Current | Delta | Delta percent | Effect |
| --- | --- | ---: | ---: | ---: | ---: | --- |
| End-to-end latency p95 (ms) | Minimize | 2,299.4543 | 2,307.7153 | +8.2609 | +0.36% | Tradeoff |
| Generation throughput (tokens/s) | Maximize | 35.3764 | 35.0959 | -0.2805 | -0.79% | Tradeoff |
| Model size (MiB) | Minimize | 1,016.8335 | 1,016.8335 | 0 | 0% | Held |
| Peak RSS (MiB) | Minimize | 1,966.4844 | 1,966.4805 | -0.0039 | -0.0001986% | Improved |
| Prompt throughput (tokens/s) | Maximize | 114.448 | 131.4565 | +17.0085 | +14.86% | Improved |
| Quality score | Maximize | 0.8333 | 0.8333 | 0 | 0% | Held |
| Time to first token p95 (ms) | Minimize | 470.4023 | 469.9681 | -0.4342 | -0.09% | Improved |

**Not comparable between these stages:** None

## Resource alternative

**Lower-resource alternative:** Q4 with KleidiAI and 512\-token micro\-batch (q4\-kleidiai\-tuned)

Compared with: Q8 generic reference (q8\-generic).

| Metric | Direction | Previous | Current | Delta | Delta percent | Effect |
| --- | --- | ---: | ---: | ---: | ---: | --- |
| End-to-end latency p95 (ms) | Minimize | 2,231.9329 | 2,307.7153 | +75.7824 | +3.4% | Tradeoff |
| Generation throughput (tokens/s) | Maximize | 38.7264 | 35.0959 | -3.6305 | -9.37% | Tradeoff |
| Model size (MiB) | Minimize | 1,806.7666 | 1,016.8335 | -789.9331 | -43.72% | Improved |
| Peak RSS (MiB) | Minimize | 3,437.5977 | 1,966.4805 | -1,471.1172 | -42.79% | Improved |
| Prompt throughput (tokens/s) | Maximize | 102.6185 | 131.4565 | +28.838 | +28.1% | Improved |
| Quality score | Maximize | 0.875 | 0.8333 | -0.0417 | -4.76% | Tradeoff |
| Time to first token p95 (ms) | Minimize | 545.3739 | 469.9681 | -75.4058 | -13.83% | Improved |

## Source and verification details

| Field | Value |
| --- | --- |
| Data source | Measured Arm64 result |
| Required source details present | Yes |
| Study type | Published benchmark |
| Runner architecture | arm64 |
| Runner reported architecture | arm64 |
| Runner CPU | Neoverse\-N2 |
| Runner CPU count | 4 |
| Runner OS | Ubuntu 24\.04 |
| Run ID | 30055662526 |
| Run attempt | 1 |
| Source repository | agrovr/ParetoPilot |
| Source revision | 8a9ddce0afa2272c4a4097fe87ef6f06cb7689a9 |
| Source workflow | \.github/workflows/candidate\-study\-arm64\.yml |
| Runtime | llama\.cpp |
| Runtime repository | https://github\.com/ggml\-org/llama\.cpp |
| Runtime revision | 67b9b0e7f6ce45d929a4411907d3c48ec719e81c |
| Model | Qwen2\.5\-1\.5B\-Instruct |
| Model repository | Qwen/Qwen2\.5\-1\.5B\-Instruct\-GGUF |
| Model revision | 91cad51170dc346986eccefdc2dd33a9da36ead9 |
| Evaluation suite | paretopilot\-qwen\-behavior\-v2 |
| Evaluation suite SHA-256 | e49c16fba32fd65c947264aef4141026ab68b1fd415ef09eeea6e8ade9a545c7 |
| Decision-summary schema | 1\.0 |
| ParetoPilot version | 1\.4\.1 |
| Input fingerprint (benchmarks\_sha256) | b95f2a2801069d1e1428faa7a3809583b26245ee7eb2e797d338cd7262d5267b |
| Input fingerprint (constraints\_sha256) | f1f9fe6a3110857ef5c7fdd78df9cd99faa6d788acbcef1bb8d1af6aecf3ca9a |

**Verification scope:** checks whether required source metadata is present; it does not independently authenticate that metadata or bind it to candidate artifacts

**Missing source details:** None

**Limits:** This result applies only to the supplied benchmark, model, runner, and workload\. It does not predict performance, energy use, or cost on other systems, and it does not establish statistical significance\.