Local inference and the real privacy boundary

Learning outcome: Map where data goes in a supposedly local setup.

Local inference runs model computation on a device or infrastructure you control. That can change the data flow, but the label “local” does not establish the behaviour of every component. A user interface may still contact remote services for updates, telemetry, search or fallback inference.

Draw the system boundary. Identify model downloads, input processing, logs, tools and network calls. Check the actual configuration and applicable organisational policy before using sensitive material. Consider device access and backups as well as transmission. A disconnected test with fictional data can help investigate dependencies, but absence of a visible error is not a complete privacy assessment.

Worked example

A fictional desktop assistant runs a small model locally but uses an online search connector for current questions. The answer includes a web result, meaning part of the task crossed the device boundary. Another configuration writes full prompts to a shared logs folder. Both behaviours matter even though model inference itself is local.

Try it yourself

Create a data-flow map for a local model, desktop interface, log folder and optional search connector. Mark which components can transmit or expose input.

You can do this exercise on paper. If using an assistant, keep to the fictional material and an approved tool.

Check your understanding

What does local inference guarantee about all connected tools?

1. Nothing by itself; their data flows need separate checking
2. They are automatically offline
3. They cannot write logs

Workbook

Download the course workbook to record your answer and compare it after the knowledge check.

Further reading

• Hugging Face: quantisation
• Hugging Face: cache strategies
• Hugging Face: LLM course