🤫 Knowledge, Research & Investigation · Note 001
The smallest machine that completes the job wins.
A pre-ship thesis on Apple M6, M5 Pro, M5 Max, and M5 Ultra for private agentic AI - and the measurement system hussh will use to find the best useful work per watt and per dollar.
We spent the first era of AI asking what a model could do.
The next era begins with a more human question: where should intelligence live?
Cloud AI has made extraordinary capability available to almost anyone. But distance has a cost. A request travels. A token becomes a recurring charge. Personal context may leave the device where it began.
For a personal agent - one that helps with the small, meaningful work of everyday life - we believe the default should be closer. Close to you. Close to your data. Close to the moment when help is needed.
That is the idea behind the new 🤫 Puppy One series: deliberate Apple-silicon configurations, guided Agent One activation, transparent permissions, and a routing principle that begins on device and goes farther only when the user approves and the job truly benefits.
Your device first.
Your permission always.
The answer in 60 seconds
Our recommended launch line.
Puppy One 32
M6 · 32GB
Personal efficiency candidatePuppy One 64
M5 Pro · 64GB
Choose for the mini footprintPuppy One 128
M5 Max · 128GB
Best professional balancePuppy One 256
M5 Ultra · 256GB
Large-model workstationPuppy One 512
M5 Ultra · 512GB
Capacity flagship, only when neededThe commercial sweet spot: Mac Studio with the 40-core M5 Max, 64GB or 128GB of unified memory, and 1TB or 2TB storage.
The efficiency candidate: Mac mini with M6 and 32GB. We expect it to lead personal, always-on workloads, but we will not call it the winner until wall-power testing is complete.
I · What changed
A five-inch efficiency story. A 512GB capacity story.
Apple's August releases widen the local-AI design space at both ends. M6 combines a 2-nanometer process, a 12-core CPU, a 12-core GPU with Neural Accelerators, dual 16-core Neural Engines, and up to 170GB/s in Mac mini. M5 Ultra combines four dies through UltraFusion, an up-to-80-core GPU, a 32-core Neural Engine, 1.2TB/s of unified-memory bandwidth, and as much as 512GB of memory in Mac Studio. Apple silicon release
| hussh tier | Apple configuration | CPU / GPU | Memory | Bandwidth | Primary job |
|---|---|---|---|---|---|
| Puppy One 32 | Mac mini · M6 | 12 / 12 cores | 32GB | 170GB/s | 4B-14B specialists; personal router |
| Puppy One 64 | Mac mini · M5 Pro | 18 / 20 cores | 64GB | 307GB/s | 27B-32B; compact professional |
| Puppy One 128 | Mac Studio · M5 Max | 18 / 40 cores | 128GB | 614GB/s | 70B; context and concurrency |
| Puppy One 256 | Mac Studio · M5 Ultra | 36 / 80 cores | 256GB | 1.2TB/s | 120B-235B; research and teams |
| Puppy One 512 | Mac Studio · M5 Ultra | 36 / 80 cores | 512GB | 1.2TB/s | 300B+ quantized experiments |
Model classes are planning ranges, not guarantees. Fit depends on architecture, weight precision, KV-cache precision, context length, runtime overhead, and concurrency. Recommended M6 32GB configurations receive 170GB/s; Apple lists 153GB/s on standard configurations. Mac mini specs · Mac Studio specs
II · Hardware is only half the product
Apple is turning the operating system into an agent surface.
Apple's software direction is as consequential as the silicon. The opportunity is not "Siri inside hussh." It is a set of supported, permissioned interfaces through which Agent One can run local models, expose its own actions, and cooperate with system experiences without pretending to be part of Apple Intelligence.
Stable foundations on macOS 26
- Foundation Models access to Apple's OS-managed on-device language model.
- App Intents for hussh-declared actions and entities in Shortcuts and Spotlight.
- Core ML for optimized packaged models.
- MLX and MLX-LM for open-weight local inference and fine-tuning on Apple silicon.
The next agentic layer
- AFM 3 Core and the sparse, multimodal 20B AFM 3 Core Advanced.
- Expanded Foundation Models APIs for images, dynamic profiles, custom providers, and eligible Private Cloud Compute access.
- Core AI for bringing custom full-scale LLMs on device.
- New Siri AI, App Intents schemas, personal context, onscreen awareness, and app actions.
AFM 3 Core Advanced is especially interesting technically: Apple describes a 20-billion-parameter sparse model that activates 1-4 billion parameters per request, storing the full model in flash and selecting experts per prompt to reduce DRAM pressure. That is a different optimization path from simply buying more memory for a dense open model. Apple Foundation Models research
It does not grant Agent One arbitrary access to Mail, Photos, Messages, Siri history, or other apps. It makes only declared actions and entities available through supported schemas. Any network tool call also changes the privacy boundary. "On device" is a property of a tested execution path, not a blanket label for the whole agent.
As of August 27, macOS 27, Core AI, expanded Foundation Models features, and Siri AI remain pre-release. Apple says macOS 27 ships this fall, with feature, language, region, entitlement, and device limits. Apple developer release · Apple Intelligence for developers
III · The metric
Token per watt is a unit.
Useful work is the product.
The inverse is joules per token. A "watt per token" dashboard is usually describing the same energy cost with imprecise units.
Decode throughput matters. But an agent also ingests prompts, embeds documents, retrieves context, invokes tools, waits on networks, checks results, and sometimes retries. A small model that produces tokens rapidly but fails the task can look efficient while wasting the user's energy.
Our north star is therefore successful tasks per joule, paired with successful tasks per dollar. Raw prompt throughput, output throughput, and token energy remain the auditable systems layer beneath it.
IV · What can be modeled before shipment
Decode is usually a memory problem. Prefill is usually a compute problem.
For a dense, quantized model generating one token at a time, the GPU repeatedly reads most model weights. A useful first-order roofline is:
For dense Q4 weights at short context, a planning assumption is roughly 0.55GB per billion parameters, plus runtime and KV cache. Observed Apple-silicon bandwidth utilization varies by chip, model, and runtime; 55-80% is a useful range, not a promise. Longer context increases KV traffic and can dominate.
That context penalty is not theoretical. A systematic MLX M3 Ultra benchmark saw Qwen 32B Q4 decline from 31.2 tok/s at 1K context to 8.5 tok/s at 128K. The right page cannot show one heroic token number; it must show a curve. MLX context study
Pre-ship batch-one decode model
llama.cpp / Metal · warm run · short context · output tokens per second. Estimates are intentionally broad. They are not PuppyBench results.
| Dense Q4 model | M6 · 170GB/s | M5 Pro · 307GB/s | M5 Max · 614GB/s | M5 Ultra · 1.2TB/s |
|---|---|---|---|---|
| 7B | 27-36est. | 53-6563.86 community run | 115-125119.92 measured | 170-235est. |
| 27B | 6-9est. | 11-16est. | 23-32est. | 44-62est. |
| 70B | Does not fit safely | 4-6est. | 8-12est. | 17-23est. |
| 120B | Does not fit | Does not fit safely | 5-7est. | 10-14est. |
M5 Max reference: maintained llama.cpp Apple-silicon table, Llama 2 7B Q4, 40-core GPU / 128GB. M5 Pro reference is a community result on an older build and is not strictly comparable. M6 and M5 Ultra rows remain empty before hardware availability. llama.cpp measurements
V · Best bang for the buck
Pay for the memory the job needs - then buy bandwidth.
Capacity is a gate. If the model and its context do not fit, the machine is not merely slower; it is the wrong machine. Once the workload fits, bandwidth and runtime determine decode speed, while Neural Accelerators and GPU compute matter heavily for prompt processing and other matrix-rich work.
M6 · 32GB / 1TB
Our personal, always-on candidate. Best for smaller specialist models, retrieval, routing, voice, and one active user. Treat 27B as an edge case, not the default.
M5 Pro · 64GB / 1TB
Capable 32B machine, but economically squeezed: the observed Apple price is only about $600 below a 40-core M5 Max / 64GB Studio with roughly twice the bandwidth. Buy Pro when the five-inch form factor matters.
M5 Max · 40 GPU / 64GB or 128GB
The strongest balance for coding, RAG, 27B-70B models, context headroom, and 1-4 concurrent flows. Move to 128GB when the model or context actually requires it.
M5 Ultra · 64 GPU / 96GB
Both Ultra GPU tiers list 1.2TB/s. The base Ultra may become the best single-stream decode-bandwidth per dollar, while supporting several smaller agents. It needs post-ship verification.
M5 Ultra · 80 GPU / 256GB or 512GB
For 120B-235B models, large KV caches, high concurrency, or compute-heavy prefill. Choose 512GB for 300B+ quantized experiments or fine-tuning - not ordinary personal-agent value.
M6 32GB/1TB: $1,699 · M5 Pro 18/20 64GB/1TB: $3,199 · M5 Max 40 GPU 64GB/1TB: $3,799 · M5 Max 40 GPU 128GB/1TB: $5,399 · M5 Ultra 80 GPU 256GB/2TB: $11,299.
Verify live before publication or sale. These are Apple hardware prices observed during research, not hussh pricing, and exclude taxes, support, setup, software, and services.VI · Token economics
Utilization and capital cost dominate electricity.
purchase price ÷ (TPS × lifetime seconds × utilization ÷ 1M)J/token × 0.27778 × electricity $/kWhAt $0.15/kWh, even 0.35-1.0 joule per token costs only about $0.015-$0.042 per million output tokens. Electricity matters for thermals, sustainability, and always-on fleets, but for a personal device, capital utilization is usually the larger economic lever.
A five-year machine at 35% inference utilization produces a capex denominator of 55.188 × TPS. That is why a memory upgrade bought "just in case" can be expensive: it may add capacity without adding single-stream bandwidth or useful throughput. The right configuration is the one you can keep productively occupied.
VII · PuppyBench
No mystery score.
No benchmark theatre.
Once shipping hardware arrives, hussh should publish a reproducible qualification book for each recommended configuration. Every result should carry the exact model, hash, precision, context, software, power method, and uncertainty.
4B / 8B / 14B / 27B / 70B dense Q4; 120B MoE; Q8 or BF16 where fit.
1K / 8K / 32K / 128K. Prompts at 512 / 4K / 32K. Generate 256 and 1,024 tokens.
1 / 2 / 4 / 8 streams. Report aggregate and per-user throughput.
Cold and warm TTFT; prefill; decode; p50/p95 inter-token latency.
Calibrated AC wall meter plus system telemetry. Report gross and idle-subtracted energy.
Ten repeats, median, p95, 95% confidence interval, plus a 30-minute sustained thermal run.
Fixed agent suite: retrieval, tool use, App Intent completion, answer acceptance, retries.
Hardware, support, electricity, cloud fallback, and successful tasks per dollar.
Publish the method beside the number. A benchmark customers cannot reproduce is an advertisement, not evidence.
VIII · The partnership boundary
Built for Apple silicon.
Independently hussh.
The public record supports a strong, truthful story: hussh can curate Apple hardware, build Agent One with public Apple developer frameworks, and serve customers who choose Mac. The three Apple releases do not name hussh or establish an Apple design or launch partnership.
Until an executed agreement supplies approved external wording, the product should say "configured by hussh on Apple silicon", not "co-developed with Apple," "mutual customers," or "Apple launch partner." That distinction protects customer trust and makes the real product achievement more credible.
It also protects the design. The goal is not to copy Apple's website or imply that anyone at Apple approved it. The goal is to apply the deeper discipline: one clear idea, ruthless editing, honest materials, and no claim that the product cannot carry.
The decision
Your agent deserves a home.
Choose Puppy One 32 for efficient everyday assistance.
Choose Puppy One 64 when the five-inch footprint and professional headroom matter.
Choose Puppy One 128 when the Mac becomes a serious local AI studio.
Choose Puppy One 256 or 512 only when model capacity, context, and data locality justify the additional capital.
The best personal supercomputer is not the one with the largest number. It is the one that quietly gets the right job done.
Your data. Your agent. Your business.Primary sources & technical evidence
Independent pre-ship analysis by hussh. Apple hardware claims and dates are attributed to Apple. hussh model-fit and performance ranges are estimates unless identified as measured. Apple, Mac, Mac mini, Mac Studio, macOS, Siri, Apple Intelligence, and related marks are trademarks of Apple Inc. hussh is independent; this report is not produced, sponsored, or endorsed by Apple.