ParetoPilot v1.1 deployment decision report

Why this decision held

The tolerance and preference policy are part of the decision, not a post-hoc explanation.

Predeclared objective tolerance

1.00%

The numeric best was Q8 generic reference at 2,231.9329 ms. The shortlist accepted values at or below 2,254.2522 ms.

The highest-preference shortlisted candidate is the numeric objective winner.

Objective values and shortlist membership
Candidate	End-to-end latency (p95)	Decision role
Q4 generic quantization	2,311.1251 ms	Outside shortlist
Q4 with KleidiAI	2,299.4543 ms	Outside shortlist
Q4 with KleidiAI and 512-token micro-batch	2,307.7153 ms	Outside shortlist
Q8 generic reference	2,231.9329 ms	Numeric best · Within tolerance · Selected

Aligned tradeoffs

The measured baseline is compared with Q4 with KleidiAI and 512-token micro-batch. “Better” and “Tradeoff” use only declared frontier directions.

End-to-end latency (p95)

Baseline2,231.9329 ms

Alternative2,307.7153 ms

Tradeoff · 3.40% higher

Time to first token (p95)

Baseline545.3739 ms

Alternative469.9681 ms

Better · 13.83% lower

Prompt processing throughput

Baseline102.6185 tok/s

Alternative131.4565 tok/s

Changed · 28.10% higher

Generation throughput

Baseline38.7264 tok/s

Alternative35.0959 tok/s

Tradeoff · 9.37% lower

Peak resident memory

Baseline3,437.5977 MiB

Alternative1,966.4805 MiB

Better · 42.79% lower

Model size

Baseline1,806.7666 MiB

Alternative1,016.8335 MiB

Better · 43.72% lower

Quality score

Baseline0.875

Alternative0.8333

Tradeoff · 4.76% lower

Deployment policy selector

Every panel below is precomputed. The browser only switches views; it does not recalculate or rank candidates.

Canonical recommendation

Q8 generic reference

Uses the submission's end-to-end p95 latency objective, 1% practical tolerance, and declared preference order.

Objective: End-to-end latency (p95)
Direction: Minimize

The highest-preference shortlisted candidate is the numeric objective winner.

Canonical submission decision.

End-to-end latency (p95)2,231.9329 ms
Time to first token (p95)545.3739 ms
Prompt processing throughput102.6185 tok/s
Generation throughput38.7264 tok/s
Peak resident memory3,437.5977 MiB

Measured load behavior

Curves appear only for supplied, validated measurements. Missing values are not interpolated or estimated.

Command and plan binding

Configuration binding: every measured load command was validated as materially identical to its canonical deployment command. Only the declared host or port binding may differ.

Load plan SHA-256: b10f61b7093f58ad6d142deafd7d6befe600d1d54a433b59c101131dd8c2d9bb

Validated load-to-deployment command bindings
Candidate	Request endpoint	Canonical parallel	Allowed differences used	Load command SHA-256	Canonical command SHA-256
Q4 generic quantization	`http://127.0.0.1:18182`	1	--port	`8557a6aca70a361ce08b2e3fa4406d71f2765d7a2d7b5ae6b61095da7b4fae03`	`0c74feec5d6df4d2070cf37c73c6199893ceb562634655282814b731bce720c9`
Q4 with KleidiAI	`http://127.0.0.1:18183`	1	--port	`f79679a2ab642d4fd14cfe9c74da40c8fef69e27e1dbde5602d50a38cc71e113`	`93bde6fa12bfc3a74719cac2a98c3f0afd447c79ba7e6ba6a29902bedc00ad87`
Q4 with KleidiAI and 512-token micro-batch	`http://127.0.0.1:18184`	1	--port	`cedca7ce6bbcd754b1c3278f605da7886be2ebf886ee2a400b67184c42ae1aa1`	`4e09666b2e6542694e0a96f3e20f27697a7f9202155f7b9c80658fd51ccc3487`
Q8 generic reference	`http://127.0.0.1:18181`	1	--port	`9a2302850253f4b86fd9b74de5cfb20afb8822683698f604d2a8de79c581cd49`	`8b5040317a38b52930ab87494bb4d02aaccf696ebe7d0826df33816a3045d74f`

Highest SLO-passing concurrency

Highest measured concurrency meeting all declared SLO gates
Field	Value
Q4 generic quantization	`1`
Q4 with kleidiai	`1`
Q4 with kleidiai and 512-token micro-batch	`1`
Q8 generic reference	`1`

Declared service-level objective

Load sweep service-level objective
Field	Value
Max e2e latency ms p95	`6500.0`
Max ttft ms p95	`2000.0`
Min completion rate	`1.0`

Measured methodology

Load sweep methodology
Field	Value
Client scheduler	`"bounded thread pool"`
Concurrency levels	`[1, 2, 4]`
Measured requests per level	`8`
Output tokens	`64`
Prompt schedule	`"round-robin"`
Prompts	[{"index": 1, "sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "text": "An engineering team is choosing an Arm64 inference configuration. Explain in two concise sentences why measured latency and memory evidence should be reviewed together."}, {"index": 2, "sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "text": "Summarize this deployment note in two concise sentences: the service must preserve response quality, keep tail latency within its declared limit, and leave enough memory for other workloads."}, {"index": 3, "sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "text": "Describe in two concise sentences how a reproducible benchmark helps a student developer make an honest deployment decision."}]
Warmup requests per level	`4`

Measured completed requests per second as concurrent request count increases.

Request throughput by concurrency data
Candidate	Concurrency	Request throughput
Q4 generic quantization	1	0.5036 req/s
Q4 generic quantization	2	0.5089 req/s
Q4 generic quantization	4	0.5087 req/s
Q4 with KleidiAI	1	0.5126 req/s
Q4 with KleidiAI	2	0.5107 req/s
Q4 with KleidiAI	4	0.5139 req/s
Q4 with KleidiAI and 512-token micro-batch	1	0.5127 req/s
Q4 with KleidiAI and 512-token micro-batch	2	0.5055 req/s
Q4 with KleidiAI and 512-token micro-batch	4	0.5151 req/s
Q8 generic reference	1	0.5457 req/s
Q8 generic reference	2	0.5492 req/s
Q8 generic reference	4	0.5391 req/s

Measured generated tokens per second as concurrent request count increases.

Generated-token throughput by concurrency data
Candidate	Concurrency	Generated-token throughput
Q4 generic quantization	1	32.2286 tok/s
Q4 generic quantization	2	32.5665 tok/s
Q4 generic quantization	4	32.5585 tok/s
Q4 with KleidiAI	1	32.8078 tok/s
Q4 with KleidiAI	2	32.682 tok/s
Q4 with KleidiAI	4	32.8879 tok/s
Q4 with KleidiAI and 512-token micro-batch	1	32.8119 tok/s
Q4 with KleidiAI and 512-token micro-batch	2	32.3519 tok/s
Q4 with KleidiAI and 512-token micro-batch	4	32.9648 tok/s
Q8 generic reference	1	34.9234 tok/s
Q8 generic reference	2	35.1494 tok/s
Q8 generic reference	4	34.5001 tok/s

Measured p95 end-to-end response latency as concurrent request count increases.

p95 end-to-end latency by concurrency data
Candidate	Concurrency	End-to-end latency (p95)
Q4 generic quantization	1	2,044.6926 ms
Q4 generic quantization	2	3,978.9609 ms
Q4 generic quantization	4	7,972.9287 ms
Q4 with KleidiAI	1	1,986.324 ms
Q4 with KleidiAI	2	3,968.8569 ms
Q4 with KleidiAI	4	7,807.2995 ms
Q4 with KleidiAI and 512-token micro-batch	1	1,984.003 ms
Q4 with KleidiAI and 512-token micro-batch	2	4,150.2731 ms
Q4 with KleidiAI and 512-token micro-batch	4	7,792.2621 ms
Q8 generic reference	1	1,915.4125 ms
Q8 generic reference	2	3,699.7596 ms
Q8 generic reference	4	7,512.7399 ms

Validated measurements at each concurrency level
Candidate	Concurrency	Measured requests	Completed requests	Request throughput	Generated-token throughput	Time to first token (p50)	Time to first token (p95)	End-to-end latency (p50)	End-to-end latency (p95)	Completion rate	Error rate	SLO status	Samples
Q4 generic quantization	1	8	8	0.5036 req/s	32.2286 tok/s	146.8574 ms	190.4137 ms	1,984.5072 ms	2,044.6926 ms	100.00%	0.00%	Met	View samples [{"completed": true, "e2e_latency_ms": 1949.6210849999898, "error": null, "finished_at_seconds": 1710.667977062, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1708.718355977, "ttft_ms": 146.68788899984975}, {"completed": true, "e2e_latency_ms": 2000.5232860000888, "error": null, "finished_at_seconds": 1712.668628779, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1710.668105493, "ttft_ms": 190.41365199996108}, {"completed": true, "e2e_latency_ms": 1928.5806919999686, "error": null, "finished_at_seconds": 1714.59731592, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1712.668735228, "ttft_ms": 104.59789899982752}, {"completed": true, "e2e_latency_ms": 2044.6926259999145, "error": null, "finished_at_seconds": 1716.642113771, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1714.597421145, "ttft_ms": 146.07854900009443}, {"completed": true, "e2e_latency_ms": 2019.6252349999213, "error": null, "finished_at_seconds": 1718.661853439, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1716.642228204, "ttft_ms": 189.46377399993253}, {"completed": true, "e2e_latency_ms": 1938.6470189999727, "error": null, "finished_at_seconds": 1720.600609379, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1718.66196236, "ttft_ms": 105.64129800013689}, {"completed": true, "e2e_latency_ms": 1968.4910539999692, "error": null, "finished_at_seconds": 1722.569210682, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1720.600719628, "ttft_ms": 147.0269989999906}, {"completed": true, "e2e_latency_ms": 2035.5482820000361, "error": null, "finished_at_seconds": 1724.604870101, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1722.569321819, "ttft_ms": 189.65193799999724}]
Q4 generic quantization	2	8	8	0.5089 req/s	32.5665 tok/s	2,108.0442 ms	2,178.0842 ms	3,924.859 ms	3,978.9609 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1950.265088000151, "error": null, "finished_at_seconds": 1734.378775629, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1732.428510541, "ttft_ms": 145.67688200008888}, {"completed": true, "e2e_latency_ms": 3948.6531970001124, "error": null, "finished_at_seconds": 1736.377427479, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1732.428774282, "ttft_ms": 2139.0681750001477}, {"completed": true, "e2e_latency_ms": 3921.9944689998556, "error": null, "finished_at_seconds": 1738.300870459, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1734.37887599, "ttft_ms": 2101.0757259998627}, {"completed": true, "e2e_latency_ms": 3863.157734999959, "error": null, "finished_at_seconds": 1740.243284453, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1736.380126718, "ttft_ms": 2065.0054479999653}, {"completed": true, "e2e_latency_ms": 3953.858683000135, "error": null, "finished_at_seconds": 1742.254827016, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1738.300968333, "ttft_ms": 2130.7733920000373}, {"completed": true, "e2e_latency_ms": 3927.7236240000093, "error": null, "finished_at_seconds": 1744.171115198, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1740.243391574, "ttft_ms": 2115.0126459999683}, {"completed": true, "e2e_latency_ms": 3905.9285629998612, "error": null, "finished_at_seconds": 1746.160865572, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1742.254937009, "ttft_ms": 2062.1976549998635}, {"completed": true, "e2e_latency_ms": 3978.9609189999737, "error": null, "finished_at_seconds": 1748.150175231, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1744.171214312, "ttft_ms": 2178.084228999978}]
Q4 generic quantization	4	8	8	0.5087 req/s	32.5585 tok/s	6,017.3187 ms	6,167.0907 ms	7,823.7666 ms	7,972.9287 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 3842.7767540001696, "error": null, "finished_at_seconds": 1759.804218509, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1755.961441755, "ttft_ms": 2046.001441000044}, {"completed": true, "e2e_latency_ms": 5926.847155000132, "error": null, "finished_at_seconds": 1761.88853919, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1755.961692035, "ttft_ms": 4031.4800959999957}, {"completed": true, "e2e_latency_ms": 1900.5368069999804, "error": null, "finished_at_seconds": 1757.862476634, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1755.961939827, "ttft_ms": 103.90798900016307}, {"completed": true, "e2e_latency_ms": 7879.752792999852, "error": null, "finished_at_seconds": 1763.842073635, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1755.962320842, "ttft_ms": 6071.819995999931}, {"completed": true, "e2e_latency_ms": 7972.928658000001, "error": null, "finished_at_seconds": 1765.835494734, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1757.862566076, "ttft_ms": 6167.090718000054}, {"completed": true, "e2e_latency_ms": 7932.512668000072, "error": null, "finished_at_seconds": 1767.73683681, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1759.804324142, "ttft_ms": 6133.627446999981}, {"completed": true, "e2e_latency_ms": 7802.7348919999895, "error": null, "finished_at_seconds": 1769.691405769, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1761.888670877, "ttft_ms": 5997.132936999833}, {"completed": true, "e2e_latency_ms": 7844.798350000019, "error": null, "finished_at_seconds": 1771.686979882, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1763.842181532, "ttft_ms": 6037.504558999899}]
Q4 with KleidiAI	1	8	8	0.5126 req/s	32.8078 tok/s	143.6856 ms	170.5038 ms	1,951.2007 ms	1,986.324 ms	100.00%	0.00%	Met	View samples [{"completed": true, "e2e_latency_ms": 1946.666162999918, "error": null, "finished_at_seconds": 1784.656093164, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1782.709427001, "ttft_ms": 144.07943500009424}, {"completed": true, "e2e_latency_ms": 1977.7977620001366, "error": null, "finished_at_seconds": 1786.633995175, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1784.656197413, "ttft_ms": 169.0082729999176}, {"completed": true, "e2e_latency_ms": 1900.2986949999467, "error": null, "finished_at_seconds": 1788.534426324, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1786.634127629, "ttft_ms": 101.44893699998647}, {"completed": true, "e2e_latency_ms": 1953.4156979998443, "error": null, "finished_at_seconds": 1790.487949318, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1788.53453362, "ttft_ms": 143.2917649999581}, {"completed": true, "e2e_latency_ms": 1986.3239559999784, "error": null, "finished_at_seconds": 1792.474412104, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1790.488088148, "ttft_ms": 169.7093440000117}, {"completed": true, "e2e_latency_ms": 1907.9663110001093, "error": null, "finished_at_seconds": 1794.382485416, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1792.474519105, "ttft_ms": 101.35752800010778}, {"completed": true, "e2e_latency_ms": 1948.985666999988, "error": null, "finished_at_seconds": 1796.331576508, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1794.382590841, "ttft_ms": 143.19726400003674}, {"completed": true, "e2e_latency_ms": 1983.8007589999052, "error": null, "finished_at_seconds": 1798.315483996, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1796.331683237, "ttft_ms": 170.50377199984723}]
Q4 with KleidiAI	2	8	8	0.5107 req/s	32.682 tok/s	2,097.2113 ms	2,133.1307 ms	3,914.6173 ms	3,968.8569 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1950.6665710000561, "error": null, "finished_at_seconds": 1808.07427089, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1806.123604319, "ttft_ms": 143.95071500007361}, {"completed": true, "e2e_latency_ms": 3939.669130999846, "error": null, "finished_at_seconds": 1810.063492956, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1806.123823825, "ttft_ms": 2118.404326000018}, {"completed": true, "e2e_latency_ms": 3932.931126000085, "error": null, "finished_at_seconds": 1812.007297258, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1808.074366132, "ttft_ms": 2097.8106229999867}, {"completed": true, "e2e_latency_ms": 3905.763395999884, "error": null, "finished_at_seconds": 1813.969350962, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1810.063587566, "ttft_ms": 2087.500675000001}, {"completed": true, "e2e_latency_ms": 3968.8569159998224, "error": null, "finished_at_seconds": 1815.976263463, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1812.007406547, "ttft_ms": 2133.1307010000273}, {"completed": true, "e2e_latency_ms": 3923.4711300000527, "error": null, "finished_at_seconds": 1817.892927477, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1813.969456347, "ttft_ms": 2111.798550000003}, {"completed": true, "e2e_latency_ms": 3845.3242040000077, "error": null, "finished_at_seconds": 1819.821679909, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1815.976355705, "ttft_ms": 2059.3017769999733}, {"completed": true, "e2e_latency_ms": 3896.660265999799, "error": null, "finished_at_seconds": 1821.78970384, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1817.893043574, "ttft_ms": 2096.611907999886}]
Q4 with KleidiAI	4	8	8	0.5139 req/s	32.8879 tok/s	5,945.6177 ms	6,008.9782 ms	7,745.128 ms	7,807.2995 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1944.0086070001144, "error": null, "finished_at_seconds": 1831.503282523, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1829.559273916, "ttft_ms": 143.28228199997284}, {"completed": true, "e2e_latency_ms": 3925.212963000149, "error": null, "finished_at_seconds": 1833.484754301, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1829.559541338, "ttft_ms": 2111.392037000087}, {"completed": true, "e2e_latency_ms": 5830.627899999854, "error": null, "finished_at_seconds": 1835.390466914, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1829.559839014, "ttft_ms": 4025.7414399998197}, {"completed": true, "e2e_latency_ms": 7783.589126999914, "error": null, "finished_at_seconds": 1837.343858705, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1829.560269578, "ttft_ms": 5972.488849000001}, {"completed": true, "e2e_latency_ms": 7807.299460999957, "error": null, "finished_at_seconds": 1839.310690585, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1831.503391124, "ttft_ms": 6008.978203999959}, {"completed": true, "e2e_latency_ms": 7729.2094389999875, "error": null, "finished_at_seconds": 1841.214060798, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1833.484851359, "ttft_ms": 5926.030214000093}, {"completed": true, "e2e_latency_ms": 7761.0466339999675, "error": null, "finished_at_seconds": 1843.151608638, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1835.390562004, "ttft_ms": 5965.2050980000695}, {"completed": true, "e2e_latency_ms": 7783.327761999999, "error": null, "finished_at_seconds": 1845.127286876, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1837.343959114, "ttft_ms": 5975.1547749999645}]
Q4 with KleidiAI and 512-token micro-batch	1	8	8	0.5127 req/s	32.8119 tok/s	157.2909 ms	176.5833 ms	1,958.5087 ms	1,984.003 ms	100.00%	0.00%	Met	View samples [{"completed": true, "e2e_latency_ms": 1943.1996179998805, "error": null, "finished_at_seconds": 1858.047414535, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1856.104214917, "ttft_ms": 143.389556999864}, {"completed": true, "e2e_latency_ms": 1971.211435999976, "error": null, "finished_at_seconds": 1860.018723052, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1858.047511616, "ttft_ms": 170.8221119999962}, {"completed": true, "e2e_latency_ms": 1913.7540710000849, "error": null, "finished_at_seconds": 1861.932586228, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1860.018832157, "ttft_ms": 102.20275300002868}, {"completed": true, "e2e_latency_ms": 1978.3992650000073, "error": null, "finished_at_seconds": 1863.911094118, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1861.932694853, "ttft_ms": 176.58332800010612}, {"completed": true, "e2e_latency_ms": 1977.528083000152, "error": null, "finished_at_seconds": 1865.888726458, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1863.911198375, "ttft_ms": 169.7822090000045}, {"completed": true, "e2e_latency_ms": 1889.4534899998234, "error": null, "finished_at_seconds": 1867.778284245, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1865.888830755, "ttft_ms": 101.47703300003741}, {"completed": true, "e2e_latency_ms": 1945.8059020000746, "error": null, "finished_at_seconds": 1869.724194668, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1867.778388766, "ttft_ms": 145.37016700001004}, {"completed": true, "e2e_latency_ms": 1984.0029950000826, "error": null, "finished_at_seconds": 1871.708301248, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1869.724298253, "ttft_ms": 169.21172800016393}]
Q4 with KleidiAI and 512-token micro-batch	2	8	8	0.5055 req/s	32.3519 tok/s	2,068.0193 ms	2,351.7369 ms	3,878.7578 ms	4,150.2731 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 4150.273077999827, "error": null, "finished_at_seconds": 1883.871618085, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1879.721345007, "ttft_ms": 2351.7368509999415}, {"completed": true, "e2e_latency_ms": 2208.4207119999064, "error": null, "finished_at_seconds": 1881.93004894, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1879.721628228, "ttft_ms": 214.0422839997882}, {"completed": true, "e2e_latency_ms": 3843.9485330000025, "error": null, "finished_at_seconds": 1885.774090843, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1881.93014231, "ttft_ms": 2045.7916169998498}, {"completed": true, "e2e_latency_ms": 3866.1812479999753, "error": null, "finished_at_seconds": 1887.737901062, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1883.871719814, "ttft_ms": 2046.5239150000798}, {"completed": true, "e2e_latency_ms": 3941.512565000039, "error": null, "finished_at_seconds": 1889.715711857, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1885.774199292, "ttft_ms": 2132.2770019999098}, {"completed": true, "e2e_latency_ms": 3891.3342719999946, "error": null, "finished_at_seconds": 1891.629362302, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1887.73802803, "ttft_ms": 2079.6453520001705}, {"completed": true, "e2e_latency_ms": 3860.6337339999754, "error": null, "finished_at_seconds": 1893.576447296, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1889.715813562, "ttft_ms": 2056.3933369999177}, {"completed": true, "e2e_latency_ms": 3917.8510120000283, "error": null, "finished_at_seconds": 1895.547310028, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1891.629459016, "ttft_ms": 2115.715507999994}]
Q4 with KleidiAI and 512-token micro-batch	4	8	8	0.5151 req/s	32.9648 tok/s	5,909.923 ms	5,987.6053 ms	7,704.7339 ms	7,792.2621 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1938.3449540000584, "error": null, "finished_at_seconds": 1905.246908952, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1903.308563998, "ttft_ms": 144.07620500014673}, {"completed": true, "e2e_latency_ms": 3919.9429020000025, "error": null, "finished_at_seconds": 1907.228757324, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1903.308814422, "ttft_ms": 2106.1574959999234}, {"completed": true, "e2e_latency_ms": 7756.85716199996, "error": null, "finished_at_seconds": 1911.065921199, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1903.309064037, "ttft_ms": 5957.664638999859}, {"completed": true, "e2e_latency_ms": 5856.797578999931, "error": null, "finished_at_seconds": 1909.166313538, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1903.309515959, "ttft_ms": 4061.5395810000336}, {"completed": true, "e2e_latency_ms": 7792.262099000027, "error": null, "finished_at_seconds": 1913.039261013, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1905.246998914, "ttft_ms": 5987.605336000115}, {"completed": true, "e2e_latency_ms": 7704.093109999803, "error": null, "finished_at_seconds": 1914.932956212, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1907.228863102, "ttft_ms": 5911.0863269997935}, {"completed": true, "e2e_latency_ms": 7705.3746330000195, "error": null, "finished_at_seconds": 1916.871776174, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1909.166401541, "ttft_ms": 5908.759608000082}, {"completed": true, "e2e_latency_ms": 7774.239778000037, "error": null, "finished_at_seconds": 1918.840284033, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1911.066044255, "ttft_ms": 5973.124458000029}]
Q8 generic reference	1	8	8	0.5457 req/s	34.9234 tok/s	166.3753 ms	221.5149 ms	1,828.9081 ms	1,915.4125 ms	100.00%	0.00%	Met	View samples [{"completed": true, "e2e_latency_ms": 1798.8625559999036, "error": null, "finished_at_seconds": 1640.790112086, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1638.99124953, "ttft_ms": 166.21581999993396}, {"completed": true, "e2e_latency_ms": 1865.428042999838, "error": null, "finished_at_seconds": 1642.655649482, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1640.790221439, "ttft_ms": 221.32033099978798}, {"completed": true, "e2e_latency_ms": 1760.4547210000874, "error": null, "finished_at_seconds": 1644.416210996, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1642.655756275, "ttft_ms": 124.19632600017394}, {"completed": true, "e2e_latency_ms": 1827.8209700001753, "error": null, "finished_at_seconds": 1646.244138551, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1644.416317581, "ttft_ms": 166.53473900009885}, {"completed": true, "e2e_latency_ms": 1880.6345320001583, "error": null, "finished_at_seconds": 1648.124881764, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1646.244247232, "ttft_ms": 221.51485600011256}, {"completed": true, "e2e_latency_ms": 1781.2798869999824, "error": null, "finished_at_seconds": 1649.90627082, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1648.124990933, "ttft_ms": 124.80181499995524}, {"completed": true, "e2e_latency_ms": 1829.995161999932, "error": null, "finished_at_seconds": 1651.736374055, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1649.906378893, "ttft_ms": 165.9717830000318}, {"completed": true, "e2e_latency_ms": 1915.4125049999493, "error": null, "finished_at_seconds": 1653.651897337, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1651.736484832, "ttft_ms": 220.88710899993202}]
Q8 generic reference	2	8	8	0.5492 req/s	35.1494 tok/s	1,997.303 ms	2,037.5929 ms	3,645.7782 ms	3,699.7596 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1817.6788560001569, "error": null, "finished_at_seconds": 1662.766442885, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1660.948764029, "ttft_ms": 166.29236800008584}, {"completed": true, "e2e_latency_ms": 3699.7595810000803, "error": null, "finished_at_seconds": 1664.648756963, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1660.948997382, "ttft_ms": 2037.5929140000153}, {"completed": true, "e2e_latency_ms": 3655.853397000101, "error": null, "finished_at_seconds": 1666.422398915, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1662.766545518, "ttft_ms": 2007.629131000158}, {"completed": true, "e2e_latency_ms": 3572.888002999889, "error": null, "finished_at_seconds": 1668.221760639, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1664.648872636, "ttft_ms": 1937.6681319999989}, {"completed": true, "e2e_latency_ms": 3657.94876699988, "error": null, "finished_at_seconds": 1670.083062646, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1666.425113879, "ttft_ms": 2016.2819259999196}, {"completed": true, "e2e_latency_ms": 3635.7030509998367, "error": null, "finished_at_seconds": 1671.857567891, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1668.22186484, "ttft_ms": 1986.976850000019}, {"completed": true, "e2e_latency_ms": 3577.882627000008, "error": null, "finished_at_seconds": 1673.661046802, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1670.083164175, "ttft_ms": 1939.8738609997963}, {"completed": true, "e2e_latency_ms": 3657.471274000045, "error": null, "finished_at_seconds": 1675.515145646, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1671.857674372, "ttft_ms": 2028.2735000000685}]
Q8 generic reference	4	8	8	0.5391 req/s	34.5001 tok/s	5,675.5386 ms	5,824.2042 ms	7,373.7089 ms	7,512.7399 ms	100.00%	0.00%	Not met	View samples [{"completed": true, "e2e_latency_ms": 1803.7393029999294, "error": null, "finished_at_seconds": 1684.617693484, "generated_tokens": 64, "index": 1, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1682.813954181, "ttft_ms": 167.27661400000216}, {"completed": true, "e2e_latency_ms": 3674.8212630000126, "error": null, "finished_at_seconds": 1686.489043627, "generated_tokens": 64, "index": 2, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1682.814222364, "ttft_ms": 2024.137742999983}, {"completed": true, "e2e_latency_ms": 5463.084967999976, "error": null, "finished_at_seconds": 1688.277522758, "generated_tokens": 64, "index": 3, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1682.81443779, "ttft_ms": 3798.668943999928}, {"completed": true, "e2e_latency_ms": 7326.948791999939, "error": null, "finished_at_seconds": 1690.141647045, "generated_tokens": 64, "index": 4, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1682.814698253, "ttft_ms": 5629.245366000077}, {"completed": true, "e2e_latency_ms": 7466.097500999922, "error": null, "finished_at_seconds": 1692.083893306, "generated_tokens": 64, "index": 5, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1684.617795805, "ttft_ms": 5746.490197999947}, {"completed": true, "e2e_latency_ms": 7420.468996999944, "error": null, "finished_at_seconds": 1693.909616441, "generated_tokens": 64, "index": 6, "prompt_index": 3, "prompt_sha256": "a48be792aff96c3b81cc8be1bc60faa0a072f1f106cf956261339b33d0d18d04", "started_at_seconds": 1686.489147444, "ttft_ms": 5721.831898999881}, {"completed": true, "e2e_latency_ms": 7466.920954999978, "error": null, "finished_at_seconds": 1695.744549106, "generated_tokens": 64, "index": 7, "prompt_index": 1, "prompt_sha256": "549face5a90122a765c1cbefe79cff73ed227a759e36590082aefbe44dcb5291", "started_at_seconds": 1688.277628151, "ttft_ms": 5797.618667000052}, {"completed": true, "e2e_latency_ms": 7512.7398800000265, "error": null, "finished_at_seconds": 1697.654487854, "generated_tokens": 64, "index": 8, "prompt_index": 2, "prompt_sha256": "5fd318ce3ceb729df9ee61f9e38c59fcd89cb885b01f4f0c8025053abb67571c", "started_at_seconds": 1690.141747974, "ttft_ms": 5824.204237999993}]

Repeat stability

Validated pass-level values show whether each measured direction repeated and how widely the candidate values spread.

Observed passes: pass-1, pass-2

Method: observed pass direction and relative spread; no significance claim.

Observed pass directions, relative deltas, and relative spread
Candidate	Metric	Direction	Pass values and deltas	Relative spread	Consistency	Overall direction
Q4 generic quantization	End-to-end latency (p95)	Lower is better	pass-12,311.1251 ms vs 2,231.9329 ms-3.55% · regressed pass-22,315.0595 ms vs 2,271.4757 ms-1.92% · regressed	0.17%	consistent	regressed
Q4 generic quantization	Generation throughput	Higher is better	pass-134.7858 tok/s vs 39.2861 tok/s-11.46% · regressed pass-235.3714 tok/s vs 38.5038 tok/s-8.14% · regressed	1.67%	consistent	regressed
Q4 generic quantization	Peak resident memory	Lower is better	pass-11,966.4609 MiB vs 3,437.5938 MiB+42.80% · improved pass-21,966.4727 MiB vs 3,437.5977 MiB+42.80% · improved	0.00%	consistent	improved
Q4 generic quantization	Prompt processing throughput	Higher is better	pass-1113.7295 tok/s vs 102.7365 tok/s+10.70% · improved pass-2113.874 tok/s vs 102.519 tok/s+11.08% · improved	0.13%	consistent	improved
Q4 generic quantization	Quality score	Higher is better	pass-10.8333 vs 0.875-4.76% · regressed pass-20.8333 vs 0.875-4.76% · regressed	0.00%	consistent	regressed
Q4 generic quantization	Time to first token (p95)	Lower is better	pass-1481.3993 ms vs 546.4947 ms+11.91% · improved pass-2483.8053 ms vs 545.3739 ms+11.29% · improved	0.50%	consistent	improved
Q4 with KleidiAI	End-to-end latency (p95)	Lower is better	pass-12,295.5086 ms vs 2,231.9329 ms-2.85% · regressed pass-22,306.7364 ms vs 2,271.4757 ms-1.55% · regressed	0.49%	consistent	regressed
Q4 with KleidiAI	Generation throughput	Higher is better	pass-135.5067 tok/s vs 39.2861 tok/s-9.62% · regressed pass-235.0159 tok/s vs 38.5038 tok/s-9.06% · regressed	1.39%	consistent	regressed
Q4 with KleidiAI	Peak resident memory	Lower is better	pass-11,966.4766 MiB vs 3,437.5938 MiB+42.79% · improved pass-21,966.4844 MiB vs 3,437.5977 MiB+42.79% · improved	0.00%	consistent	improved
Q4 with KleidiAI	Prompt processing throughput	Higher is better	pass-1114.532 tok/s vs 102.7365 tok/s+11.48% · improved pass-2114.425 tok/s vs 102.519 tok/s+11.61% · improved	0.09%	consistent	improved
Q4 with KleidiAI	Quality score	Higher is better	pass-10.8333 vs 0.875-4.76% · regressed pass-20.8333 vs 0.875-4.76% · regressed	0.00%	consistent	regressed
Q4 with KleidiAI	Time to first token (p95)	Lower is better	pass-1467.1517 ms vs 546.4947 ms+14.52% · improved pass-2471.8408 ms vs 545.3739 ms+13.48% · improved	1.00%	consistent	improved
Q4 with KleidiAI and 512-token micro-batch	End-to-end latency (p95)	Lower is better	pass-12,312.7413 ms vs 2,231.9329 ms-3.62% · regressed pass-22,307.7153 ms vs 2,271.4757 ms-1.60% · regressed	0.22%	consistent	regressed
Q4 with KleidiAI and 512-token micro-batch	Generation throughput	Higher is better	pass-134.9091 tok/s vs 39.2861 tok/s-11.14% · regressed pass-235.1905 tok/s vs 38.5038 tok/s-8.61% · regressed	0.80%	consistent	regressed
Q4 with KleidiAI and 512-token micro-batch	Peak resident memory	Lower is better	pass-11,966.4805 MiB vs 3,437.5938 MiB+42.79% · improved pass-21,966.4727 MiB vs 3,437.5977 MiB+42.80% · improved	0.00%	consistent	improved
Q4 with KleidiAI and 512-token micro-batch	Prompt processing throughput	Higher is better	pass-1131.516 tok/s vs 102.7365 tok/s+28.01% · improved pass-2131.361 tok/s vs 102.519 tok/s+28.13% · improved	0.12%	consistent	improved
Q4 with KleidiAI and 512-token micro-batch	Quality score	Higher is better	pass-10.8333 vs 0.875-4.76% · regressed pass-20.8333 vs 0.875-4.76% · regressed	0.00%	consistent	regressed
Q4 with KleidiAI and 512-token micro-batch	Time to first token (p95)	Lower is better	pass-1467.41 ms vs 546.4947 ms+14.47% · improved pass-2470.2895 ms vs 545.3739 ms+13.77% · improved	0.61%	consistent	improved
Q8 generic reference	End-to-end latency (p95)	Lower is better	pass-12,231.9329 ms vs 2,231.9329 ms+0.00% · no change pass-22,271.4757 ms vs 2,271.4757 ms+0.00% · no change	1.76%	no change	no change
Q8 generic reference	Generation throughput	Higher is better	pass-139.2861 tok/s vs 39.2861 tok/s+0.00% · no change pass-238.5038 tok/s vs 38.5038 tok/s+0.00% · no change	2.01%	no change	no change
Q8 generic reference	Peak resident memory	Lower is better	pass-13,437.5938 MiB vs 3,437.5938 MiB+0.00% · no change pass-23,437.5977 MiB vs 3,437.5977 MiB+0.00% · no change	0.00%	no change	no change
Q8 generic reference	Prompt processing throughput	Higher is better	pass-1102.7365 tok/s vs 102.7365 tok/s+0.00% · no change pass-2102.519 tok/s vs 102.519 tok/s+0.00% · no change	0.21%	no change	no change
Q8 generic reference	Quality score	Higher is better	pass-10.875 vs 0.875+0.00% · no change pass-20.875 vs 0.875+0.00% · no change	0.00%	no change	no change
Q8 generic reference	Time to first token (p95)	Lower is better	pass-1546.4947 ms vs 546.4947 ms+0.00% · no change pass-2545.3739 ms vs 545.3739 ms+0.00% · no change	0.21%	no change	no change

Interpretation limit: consistency and relative spread describe these observed passes only. They are not a statistical-significance claim.

p95 end-to-end latency vs generation throughput

This accurately scoped two-dimensional view complements, but does not replace, the full constraint and frontier analysis.

Each labeled point is one measured candidate. Left is lower p95 end-to-end latency; higher is greater generation throughput. This is a two-metric view, not the complete multi-objective frontier.

p95 end-to-end latency vs generation throughput data
Candidate	p95 end-to-end latency	Generation throughput	Decision role
Q4 generic quantization	2,311.1251 ms	35.0123 tok/s	Declared frontier
Q4 with KleidiAI	2,299.4543 ms	35.3764 tok/s	Declared frontier
Q4 with KleidiAI and 512-token micro-batch	2,307.7153 ms	35.0959 tok/s	Declared frontier
Q8 generic reference	2,231.9329 ms	38.7264 tok/s	Baseline · Selected · Declared frontier

Full evidence table

Missing values remain explicitly unmeasured. Rejection reasons are copied from the primary recommendation record.

All candidate measurements, parameters, statuses, and rejection reasons
Candidate	Decision status	Parameters	End-to-end latency (p95)	Time to first token (p95)	Prompt processing throughput	Generation throughput	Peak resident memory	Model size	Quality score	Constraint result
Q4 generic quantization`q4-generic`	Declared frontier · Eligible	View configuration {"configuration": {"attribution_stage": "quantization", "batch_size": 512, "context_size": 2048, "kleidiai": false, "runtime_tuned": false, "threads": 4, "ubatch_size": 128}, "deployment_argv": ["./.candidate-build/generic/bin/llama-server", "--model", "./.candidate-models/qwen2.5-1.5b-instruct-q4_0.gguf", "--threads", "4", "--threads-batch", "4", "--batch-size", "512", "--ubatch-size", "128", "--ctx-size", "2048", "--parallel", "1", "--n-gpu-layers", "0", "-lv", "4", "--host", "127.0.0.1", "--port", "18082"], "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}, "runtime": {"name": "llama.cpp", "repository": "https://github.com/ggml-org/llama.cpp", "revision": "67b9b0e7f6ce45d929a4411907d3c48ec719e81c"}}	2,311.1251 ms	483.1131 ms	113.821 tok/s	35.0123 tok/s	1,966.4727 MiB	1,016.8335 MiB	0.8333	No declared constraint violations.
Q4 with KleidiAI`q4-kleidiai`	Declared frontier · Eligible	View configuration {"configuration": {"attribution_stage": "arm-kernel", "batch_size": 512, "context_size": 2048, "kleidiai": true, "runtime_tuned": false, "threads": 4, "ubatch_size": 128}, "deployment_argv": ["./.candidate-build/kleidiai/bin/llama-server", "--model", "./.candidate-models/qwen2.5-1.5b-instruct-q4_0.gguf", "--threads", "4", "--threads-batch", "4", "--batch-size", "512", "--ubatch-size", "128", "--ctx-size", "2048", "--parallel", "1", "--n-gpu-layers", "0", "-lv", "4", "--host", "127.0.0.1", "--port", "18083"], "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}, "runtime": {"name": "llama.cpp", "repository": "https://github.com/ggml-org/llama.cpp", "revision": "67b9b0e7f6ce45d929a4411907d3c48ec719e81c"}}	2,299.4543 ms	470.4023 ms	114.448 tok/s	35.3764 tok/s	1,966.4844 MiB	1,016.8335 MiB	0.8333	No declared constraint violations.
Q4 with KleidiAI and 512-token micro-batch`q4-kleidiai-tuned`	Declared frontier · Eligible	View configuration {"configuration": {"attribution_stage": "runtime-tuning", "batch_size": 512, "context_size": 2048, "kleidiai": true, "runtime_tuned": true, "threads": 4, "ubatch_size": 512}, "deployment_argv": ["./.candidate-build/kleidiai/bin/llama-server", "--model", "./.candidate-models/qwen2.5-1.5b-instruct-q4_0.gguf", "--threads", "4", "--threads-batch", "4", "--batch-size", "512", "--ubatch-size", "512", "--ctx-size", "2048", "--parallel", "1", "--n-gpu-layers", "0", "-lv", "4", "--host", "127.0.0.1", "--port", "18084"], "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}, "runtime": {"name": "llama.cpp", "repository": "https://github.com/ggml-org/llama.cpp", "revision": "67b9b0e7f6ce45d929a4411907d3c48ec719e81c"}}	2,307.7153 ms	469.9681 ms	131.4565 tok/s	35.0959 tok/s	1,966.4805 MiB	1,016.8335 MiB	0.8333	No declared constraint violations.
Q8 generic reference`q8-generic`	Selected · Baseline · Declared frontier · Eligible	View configuration {"configuration": {"attribution_stage": "reference", "batch_size": 512, "context_size": 2048, "kleidiai": false, "runtime_tuned": false, "threads": 4, "ubatch_size": 128}, "deployment_argv": ["./.candidate-build/generic/bin/llama-server", "--model", "./.candidate-models/qwen2.5-1.5b-instruct-q8_0.gguf", "--threads", "4", "--threads-batch", "4", "--batch-size", "512", "--ubatch-size", "128", "--ctx-size", "2048", "--parallel", "1", "--n-gpu-layers", "0", "-lv", "4", "--host", "127.0.0.1", "--port", "18081"], "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q8_0.gguf", "quantization": "Q8_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "d7efb072e7724d25048a4fda0a3e10b04bdef5d06b1403a1c93bd9f1240a63c8", "size_bytes": 1894532128}, "runtime": {"name": "llama.cpp", "repository": "https://github.com/ggml-org/llama.cpp", "revision": "67b9b0e7f6ce45d929a4411907d3c48ec719e81c"}}	2,231.9329 ms	545.3739 ms	102.6185 tok/s	38.7264 tok/s	3,437.5977 MiB	1,806.7666 MiB	0.875	No declared constraint violations.

Trust and reproduction

Hashes and source metadata anchor this deterministic document. No generation timestamp or network-loaded asset is added.

Source anchors

Benchmark SHA-256: b95f2a2801069d1e1428faa7a3809583b26245ee7eb2e797d338cd7262d5267b
Recommendation SHA-256: e23b50ddd02ed72200417ddb146215c3aa995fb9c8c1a98faa64ee6ba925c7a4
Policy profiles SHA-256: 6fe9d423a5c41bd48c74fa255c19fb174c8d09e718ecd45d5ab9ff8d23508c37
Load sweep SHA-256: 923b3222440314644c5e87b326d7d7fb1a394b9d688262ffa636d6ef275e9dfd
Stability summary SHA-256: e3d1ee557005efb076ed1a34ba38fb52e3378b1eead8a8f745a85f25773712a3
Benchmark schema: 1.0
ParetoPilot version: 1.1.0
Baseline ID: q8-generic
Selected ID: q8-generic

Selected deployment argv

The command below reproduces only the selected server configuration. It was rendered from the validated argv vector; no flags or paths were guessed.

./.candidate-build/generic/bin/llama-server --model ./.candidate-models/qwen2.5-1.5b-instruct-q8_0.gguf --threads 4 --threads-batch 4 --batch-size 512 --ubatch-size 128 --ctx-size 2048 --parallel 1 --n-gpu-layers 0 -lv 4 --host 127.0.0.1 --port 18081

View full source metadata

Source-supplied benchmark metadata
Field	Value
Candidate evidence	{"q4-generic": {"artifacts": {"llama_summary": {"path": "candidates/q4-generic/llama-summary.json", "sha256": "04e8ad1043a3b155a42ac32a90a7dcde434bd6aac829cb540e883c8084831c42"}, "resource_measurement": {"path": "candidates/q4-generic/resource-measurement.json", "sha256": "3e87fd5ba9ffef213df355d185955b47d9907c59e6479fbedca375ceb73ce1e6"}, "server_evaluation": {"path": "candidates/q4-generic/server-evaluation.json", "sha256": "2f390312f2d0c8079ca6e86d6f74507a45648018cd023f1d50e271f7fec8e789"}, "server_evaluation_pass_1": {"path": "candidates/q4-generic/server-evaluation-pass-1.json", "sha256": "20b90cf334e211b2de4faaee98fe304f7a601e90026bd778d13249e20b3ec0b8"}, "server_evaluation_pass_2": {"path": "candidates/q4-generic/server-evaluation-pass-2.json", "sha256": "cdd29a077d46c05c26cade5dd1e74c4a8fc1db2d49a64d8f82dcda4a9fecaf6a"}, "server_stderr_pass_1": {"path": "candidates/q4-generic/server-pass-1.stderr.log", "sha256": "35f467ec737f5860c02b5a51e1b640bd34fc90fb8bf56cee6e4b94b844eb3baf"}, "server_stderr_pass_2": {"path": "candidates/q4-generic/server-pass-2.stderr.log", "sha256": "debbf7fe3683871cfbaa437c6dc6272746943105f43e24ba1b78aea98984f72b"}, "server_time_pass_1": {"path": "candidates/q4-generic/server-time-pass-1.txt", "sha256": "3cde7534471d26a13600a0d6a68799eeef3fcff6d3c5129083f52c9a6c4ca2d9"}, "server_time_pass_2": {"path": "candidates/q4-generic/server-time-pass-2.txt", "sha256": "d112bd9f197fcde159204138e132c3aca696e75643c53d3d148aa5a2e80df4a0"}, "throughput_command": {"path": "candidates/q4-generic/throughput-command.json", "sha256": "fdd7e4954c2c3d3e924dcc51b950bedcda74da8c15855d9d83a94b2301e449d5"}, "throughput_pass_1": {"path": "candidates/q4-generic/raw/pass-1.jsonl", "sha256": "9d545db9509886dd94a78de0421826b087eb3f61ca966fb2ff3fdc7919a56c48"}, "throughput_pass_2": {"path": "candidates/q4-generic/raw/pass-2.jsonl", "sha256": "5fca70848fc11e151dd486b61d90e6c89fffea7cb9f83fe680a012c07d3d765e"}, "throughput_settings": {"path": "candidates/q4-generic/settings.json", "sha256": "a0e58888c94cc186216af765357eca03d8a3753b92216efc1321a1985129bc18"}, "throughput_stderr_pass_1": {"path": "candidates/q4-generic/raw/pass-1.stderr.log", "sha256": "98b3e0afd84fa5c8a8d251891a3a2ed0441bd5ad54518e92023632faf93e4055"}, "throughput_stderr_pass_2": {"path": "candidates/q4-generic/raw/pass-2.stderr.log", "sha256": "492ccbb636b2eb243e0c1525c499fcf67a97ebaf87757375698f1e1e3bb96ec9"}}, "metric_sources": {"e2e_latency_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/e2e_latency_ms_p95"}, "generation_tps": {"artifact": "llama_summary", "json_pointer": "/tests/tg/tokens_per_second/median"}, "model_size_mib": {"artifact": "manifest", "json_pointer": "/candidates//model/size_bytes (divided by 1048576)"}, "peak_rss_mib": {"artifact": "resource_measurement", "json_pointer": "/maximum_resident_set_kbytes (divided by 1024)"}, "prompt_tps": {"artifact": "llama_summary", "json_pointer": "/tests/pp/tokens_per_second/median"}, "quality_score": {"artifact": "server_evaluation", "json_pointer": "/quality/score"}, "ttft_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/ttft_ms_p95"}}, "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}}, "q4-kleidiai": {"artifacts": {"llama_summary": {"path": "candidates/q4-kleidiai/llama-summary.json", "sha256": "b95143c12f9ef2622eef31b041a0334c8f1461bc9fd76dc32b403f531f8a3460"}, "resource_measurement": {"path": "candidates/q4-kleidiai/resource-measurement.json", "sha256": "4b6e97b9f695fc5bc6a4b2e44560c19ae1c88169c718783474e67714edccea09"}, "server_evaluation": {"path": "candidates/q4-kleidiai/server-evaluation.json", "sha256": "7ff8571dcbd437d763cc196244a5b8df5e1c49ca6824898d8ebb548ae8a093df"}, "server_evaluation_pass_1": {"path": "candidates/q4-kleidiai/server-evaluation-pass-1.json", "sha256": "8571713dd511b4f40379dd38d77d8ee4a3993348382edac0e34a3ae679b0292e"}, "server_evaluation_pass_2": {"path": "candidates/q4-kleidiai/server-evaluation-pass-2.json", "sha256": "1290479aec1da03b4272f5b9eadfa277818c9b8b5ff45396cfa41cd5afccf67c"}, "server_stderr_pass_1": {"path": "candidates/q4-kleidiai/server-pass-1.stderr.log", "sha256": "7338bd441a8ba9afc7c1693f188e1e885bedb7becbf4ab5863ba8eee60514b8c"}, "server_stderr_pass_2": {"path": "candidates/q4-kleidiai/server-pass-2.stderr.log", "sha256": "9c9ebfbdd26452e35d6c55d2d08fb2c49be5f51be0d42ab1fb95576f5c58ea08"}, "server_time_pass_1": {"path": "candidates/q4-kleidiai/server-time-pass-1.txt", "sha256": "ebba9031a349f4094d32a428b853fcae56dbbcb2643b7a55ba68713a7410815a"}, "server_time_pass_2": {"path": "candidates/q4-kleidiai/server-time-pass-2.txt", "sha256": "c570ff52671d5d30fd4d5b1d5bc08b5e1baab01b98e79cb6458b5ba11588cfef"}, "throughput_command": {"path": "candidates/q4-kleidiai/throughput-command.json", "sha256": "5bd7daafc6e8c2da09ddbc3921ec7a76703b0c92e3bea0da3d048673dbf7a3f5"}, "throughput_pass_1": {"path": "candidates/q4-kleidiai/raw/pass-1.jsonl", "sha256": "1da5bf0b3f3cca835634619a3ea14a3d3b6dd8e5974ec6a8dfb2049121e4002e"}, "throughput_pass_2": {"path": "candidates/q4-kleidiai/raw/pass-2.jsonl", "sha256": "4061217d3f69ddf91852a7fcc84931f8b098dd758030692abd7087069c2345e4"}, "throughput_settings": {"path": "candidates/q4-kleidiai/settings.json", "sha256": "69c9e507a996bf87c8c7dd4203462598498edc12525b5695b02ea434e3931ccf"}, "throughput_stderr_pass_1": {"path": "candidates/q4-kleidiai/raw/pass-1.stderr.log", "sha256": "8d9e4b8f7efbbdf087e7d5d55a82eb94eafe78dbef9fe66de402dc524c0e6bf6"}, "throughput_stderr_pass_2": {"path": "candidates/q4-kleidiai/raw/pass-2.stderr.log", "sha256": "80cd4d7bce8238b1a0439131dcab2b2419928462b2cca4e1b4a0975efc39d1e5"}}, "metric_sources": {"e2e_latency_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/e2e_latency_ms_p95"}, "generation_tps": {"artifact": "llama_summary", "json_pointer": "/tests/tg/tokens_per_second/median"}, "model_size_mib": {"artifact": "manifest", "json_pointer": "/candidates//model/size_bytes (divided by 1048576)"}, "peak_rss_mib": {"artifact": "resource_measurement", "json_pointer": "/maximum_resident_set_kbytes (divided by 1024)"}, "prompt_tps": {"artifact": "llama_summary", "json_pointer": "/tests/pp/tokens_per_second/median"}, "quality_score": {"artifact": "server_evaluation", "json_pointer": "/quality/score"}, "ttft_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/ttft_ms_p95"}}, "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}}, "q4-kleidiai-tuned": {"artifacts": {"llama_summary": {"path": "candidates/q4-kleidiai-tuned/llama-summary.json", "sha256": "05ec5eb7ab35fcbc5efff2f3843d4050c64074ba5bf9380c3eee1d0b1bd9fb03"}, "resource_measurement": {"path": "candidates/q4-kleidiai-tuned/resource-measurement.json", "sha256": "75e17b27f624f00c0ec597d23e74a7429294bc4e4ada57fc2df37c2edba6b910"}, "server_evaluation": {"path": "candidates/q4-kleidiai-tuned/server-evaluation.json", "sha256": "b240649cf666dcf2e5e8a121806282a96b77460c914868a3ddca99f7a658c4f3"}, "server_evaluation_pass_1": {"path": "candidates/q4-kleidiai-tuned/server-evaluation-pass-1.json", "sha256": "3a1e1559d30946d6d1edbb99edf6af716414cfcd1c4a320fefe0352e274ed61c"}, "server_evaluation_pass_2": {"path": "candidates/q4-kleidiai-tuned/server-evaluation-pass-2.json", "sha256": "0c8683dae11abe889caefff3687caa2b2000f95d6af5fb7c3a7c278feb27da1b"}, "server_stderr_pass_1": {"path": "candidates/q4-kleidiai-tuned/server-pass-1.stderr.log", "sha256": "0eea701e6ae7a432fdd901af2a6f6988e68558a99ec2dc3da7bc90e68c997c69"}, "server_stderr_pass_2": {"path": "candidates/q4-kleidiai-tuned/server-pass-2.stderr.log", "sha256": "b1e740f0c8df535772504ebf797f75b4ce09b718e5c3a9e5c11c9ec998a96403"}, "server_time_pass_1": {"path": "candidates/q4-kleidiai-tuned/server-time-pass-1.txt", "sha256": "0822190409d7b22a529d2ae770e70d680b158812508d15045edff07f27f0649a"}, "server_time_pass_2": {"path": "candidates/q4-kleidiai-tuned/server-time-pass-2.txt", "sha256": "8a5d84c7035c05c368a9eff00b686bd57489b36581b30e57ea2d44c19ec8b2bf"}, "throughput_command": {"path": "candidates/q4-kleidiai-tuned/throughput-command.json", "sha256": "f44706e79d6532205f5fd53abf00479995cab9abd86f0b6c6e4e7309bb481b12"}, "throughput_pass_1": {"path": "candidates/q4-kleidiai-tuned/raw/pass-1.jsonl", "sha256": "c4c36b7900ad4d5cefb4ed2e09109eaef51a60a9bdefbeb2a78df3a0e2c60853"}, "throughput_pass_2": {"path": "candidates/q4-kleidiai-tuned/raw/pass-2.jsonl", "sha256": "21bce7cdcbcc1cb8ec9599b0fbe76a62642dfbf72b3fcd97dadd7203a5a6dabc"}, "throughput_settings": {"path": "candidates/q4-kleidiai-tuned/settings.json", "sha256": "06698a361f4877d74c3ace0d1c33ad70a130525abd7b005d4b80a4b0287d844f"}, "throughput_stderr_pass_1": {"path": "candidates/q4-kleidiai-tuned/raw/pass-1.stderr.log", "sha256": "50b3cb922958d95b2322301eb0b83c6f93333365cb40fcc7b7884d3e11217637"}, "throughput_stderr_pass_2": {"path": "candidates/q4-kleidiai-tuned/raw/pass-2.stderr.log", "sha256": "e7702c6b5ac5a25fffb390a99571f46d5b4a4202a95f3fa35598634199371af5"}}, "metric_sources": {"e2e_latency_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/e2e_latency_ms_p95"}, "generation_tps": {"artifact": "llama_summary", "json_pointer": "/tests/tg/tokens_per_second/median"}, "model_size_mib": {"artifact": "manifest", "json_pointer": "/candidates//model/size_bytes (divided by 1048576)"}, "peak_rss_mib": {"artifact": "resource_measurement", "json_pointer": "/maximum_resident_set_kbytes (divided by 1024)"}, "prompt_tps": {"artifact": "llama_summary", "json_pointer": "/tests/pp/tokens_per_second/median"}, "quality_score": {"artifact": "server_evaluation", "json_pointer": "/quality/score"}, "ttft_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/ttft_ms_p95"}}, "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q4_0.gguf", "quantization": "Q4_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "dcd819ff094852c38faba6873d8ff0c9d51eadb2844539e52042ae5d647bbfdb", "size_bytes": 1066227232}}, "q8-generic": {"artifacts": {"llama_summary": {"path": "candidates/q8-generic/llama-summary.json", "sha256": "5a64bedf1b1d295b1a56bf19cb57786282375cb51ab939805be4188720f74219"}, "resource_measurement": {"path": "candidates/q8-generic/resource-measurement.json", "sha256": "d184d590679da493d3ddd1156faafba286e8705697957994ed411bada765adea"}, "server_evaluation": {"path": "candidates/q8-generic/server-evaluation.json", "sha256": "e9b8d1359b838edc4621499c4fc506efaeecac64a39688052d4ac274322040c1"}, "server_evaluation_pass_1": {"path": "candidates/q8-generic/server-evaluation-pass-1.json", "sha256": "a30ef003fe8bdc0dc92db4f0ffac21653fd853bd0d689c315828efc5190731f1"}, "server_evaluation_pass_2": {"path": "candidates/q8-generic/server-evaluation-pass-2.json", "sha256": "629be14c1a7307d8bee12816d03a35002d75a495d4276845f2c99581abcc1fa1"}, "server_stderr_pass_1": {"path": "candidates/q8-generic/server-pass-1.stderr.log", "sha256": "8676225e38654ca474cb59c737b198823fc916a41df7fb1c7b8be1bfde0fd0af"}, "server_stderr_pass_2": {"path": "candidates/q8-generic/server-pass-2.stderr.log", "sha256": "3ea47bca3d9497c0d1f7e628e70d134b01b29da03f99f4305cf0c522979a75f0"}, "server_time_pass_1": {"path": "candidates/q8-generic/server-time-pass-1.txt", "sha256": "1d54a5a189731eff0880929d1434b1771f60eb6adb47f36e5cd079d948404d78"}, "server_time_pass_2": {"path": "candidates/q8-generic/server-time-pass-2.txt", "sha256": "ad5d7e91fea9e0839e1de3239bdba20d50e097024d44b8bfc221f6a418632513"}, "throughput_command": {"path": "candidates/q8-generic/throughput-command.json", "sha256": "3e34dc7419db28305e96ae686a16c9db5583e864e2cc5fd37e5d85d356eb1504"}, "throughput_pass_1": {"path": "candidates/q8-generic/raw/pass-1.jsonl", "sha256": "c270df39f12aef4e0643520ae3587e1e52e131358298a3229086bc4d44f53bf7"}, "throughput_pass_2": {"path": "candidates/q8-generic/raw/pass-2.jsonl", "sha256": "2f34053759527077229f1c06765e0cbea87147174a3b084e69893c4902000bca"}, "throughput_settings": {"path": "candidates/q8-generic/settings.json", "sha256": "5c344fb0afaa527bf17447eef11362d27188a791b861fff72c135ddef17a571e"}, "throughput_stderr_pass_1": {"path": "candidates/q8-generic/raw/pass-1.stderr.log", "sha256": "310fb48add49717b7a11b4cc84291f9d83ba65c3b253ede2163de5306c7a813e"}, "throughput_stderr_pass_2": {"path": "candidates/q8-generic/raw/pass-2.stderr.log", "sha256": "667ecb7ef19c636126b2b11544bc06a83500cae9295b7f7f5bbefff5ecab6a64"}}, "metric_sources": {"e2e_latency_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/e2e_latency_ms_p95"}, "generation_tps": {"artifact": "llama_summary", "json_pointer": "/tests/tg/tokens_per_second/median"}, "model_size_mib": {"artifact": "manifest", "json_pointer": "/candidates//model/size_bytes (divided by 1048576)"}, "peak_rss_mib": {"artifact": "resource_measurement", "json_pointer": "/maximum_resident_set_kbytes (divided by 1024)"}, "prompt_tps": {"artifact": "llama_summary", "json_pointer": "/tests/pp/tokens_per_second/median"}, "quality_score": {"artifact": "server_evaluation", "json_pointer": "/quality/score"}, "ttft_ms_p95": {"artifact": "server_evaluation", "json_pointer": "/latency/ttft_ms_p95"}}, "model": {"family": "Qwen2.5-1.5B-Instruct", "filename": "qwen2.5-1.5b-instruct-q8_0.gguf", "quantization": "Q8_0", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9", "sha256": "d7efb072e7724d25048a4fda0a3e10b04bdef5d06b1403a1c93bd9f1240a63c8", "size_bytes": 1894532128}}}
Classification	`"canonical"`
Evaluation suite	`{"id": "paretopilot-qwen-behavior-v2", "sha256": "e49c16fba32fd65c947264aef4141026ab68b1fd415ef09eeea6e8ade9a545c7"}`
Experiment id	`"paretopilot-arm64-candidate-study-v1"`
Manifest	`{"filename": "manifest.json", "sha256": "7a8760e2950b7cc0c34c9377a57bee8dc41b58a229866f07e5cf55ffc1c91c82"}`
Model family	`{"name": "Qwen2.5-1.5B-Instruct", "repository": "Qwen/Qwen2.5-1.5B-Instruct-GGUF", "revision": "91cad51170dc346986eccefdc2dd33a9da36ead9"}`
Optimization library	`{"name": "KleidiAI", "repository": "https://github.com/ARM-software/kleidiai", "source_archive_sha256": "9348b969e042d8890a54b01a463dbe71f5a4c074b5329e9c26a85ef3b68aa19b", "source_archive_size_bytes": 2466038, "version": "v1.24.0"}`
Runtime	`{"name": "llama.cpp", "repository": "https://github.com/ggml-org/llama.cpp", "revision": "67b9b0e7f6ce45d929a4411907d3c48ec719e81c"}`
Source	`{"generated_at_utc": "2026-07-24T00:46:28Z", "repository": "agrovr/ParetoPilot", "revision": "8a9ddce0afa2272c4a4097fe87ef6f06cb7689a9", "run_attempt": 1, "run_id": "30055662526", "runner": {"architecture": "arm64", "cpu": "Neoverse-N2", "cpu_count": 4, "os": "Ubuntu 24.04"}, "workflow": ".github/workflows/candidate-study-arm64.yml"}`

Reproduction contract

Verify the evidence archive and recorded source hashes.
Rebuild the benchmark set and primary recommendation from the validated inputs.
Render into a fresh path and compare the resulting artifact hash.

The v1.1 renderer is additive. The v1.0 renderer remains available for replaying the published v1.0 evidence.

Why this decision held

Aligned tradeoffs

Deployment policy selector

Q8 generic reference

Q4 generic quantization

Q4 with KleidiAI and 512-token micro-batch

Q4 with KleidiAI and 512-token micro-batch

Q8 generic reference

Measured load behavior

Command and plan binding

Highest SLO-passing concurrency

Declared service-level objective

Measured methodology

Repeat stability

p95 end-to-end latency vs generation throughput

Full evidence table

Trust and reproduction

Source anchors

Selected deployment argv

Reproduction contract