Which offline AI models are worth running on a student laptop

The assumption that running AI locally requires expensive hardware is roughly two years out of date. Quantisation and smaller, better-trained models have moved the floor down far enough that a mid-range school laptop is a genuine target.

What has not changed is that model choice matters more than it does in the cloud, because you are the one paying for the memory.

The only three numbers that matter

A 4-bit model in the 3-to-8 billion parameter range is the sweet spot for a laptop. It fits comfortably in memory, answers fast, and handles the tasks students actually bring it.

• Parameter count — roughly how much the model knows. More is better and heavier.
• Quantisation — the precision the weights are stored at. 4-bit roughly quarters the memory cost versus 16-bit, at a quality cost most students will not notice on everyday tasks.
• Context length — how much text it can hold at once. This is what determines whether you can feed it a whole chapter or only a few paragraphs.

What local models are already good at

• Summarising a chapter or a transcript.
• Explaining a concept at a chosen level, repeatedly and patiently.
• Extracting text from a photograph of handwriting or a worksheet.
• Generating practice questions from supplied material.
• Rewriting for clarity, tone or reading level.
• Small, well-specified programs and explaining existing code.
• Translation and language-practice correction.

Where the cloud still wins

This is a real gap, not a marketing caveat. The practical answer is a default of local with a deliberate escalation path — which is how Grout is arranged, with on-device models by default and optional Premium cloud models when a task warrants one.

• Long multi-step reasoning where a small error compounds.
• Anything requiring knowledge of recent events — every local model has a cutoff and no browser.
• Very long documents, where a large context window is the whole game.
• Niche specialist domains that only a very large model has absorbed.

What a given machine can do

Unified memory on Apple Silicon punches above its number here, because the GPU can address the whole pool. A 16 GB MacBook Air is a surprisingly strong local AI machine.

• 8 GB RAM: small 4-bit text models, vision OCR, light image generation. Usable, and offline.
• 16 GB RAM: medium models with a longer context, full vision and comfortable image generation. This is the sweet spot.
• Apple Silicon or a discrete GPU: large models, fast generation and hardware-accelerated video export.

Practical setup notes

• Download models on a fast network before you need them — this is the one step that needs a connection.
• Plug in for long generation runs; local inference uses the battery hard.
• Keep one small fast model for quick questions and one larger one for real work, rather than running everything through the biggest thing that fits.
• On a school device, ask IT to pre-load the models onto the laptop image so devices arrive ready.