AI ToolsACADEMY
AI Tools Academy
0 / 117 (0%)

Local models and inference engineering

Finishing the course

Capstone · 30 min

Your final assignment

Produce a deployment comparison using measured constraints rather than model-size slogans. Use the fictional examples from this course, or a non-sensitive scenario you are authorised to work on. Do not submit personal workplace records to complete the course.

Build your evidence pack

  1. Local inference and the real privacy boundary: Create a data-flow map for a local model, desktop interface, log folder and optional search connector. Mark which components can transmit or expose input.
  2. Quantisation and memory estimates: Calculate weight-only storage for three billion parameters at eight bits and sixteen bits. List three reasons actual memory use can be higher.
  3. KV caches and long-context memory: Design a test matrix with short and long inputs, one and several concurrent requests, and a fixed output limit. Record peak memory and whether requests complete.
  4. Latency, throughput and the waiting user: Write a measurement plan for live help and overnight document processing. Choose two metrics for each and explain why the priorities differ.
  5. Adapters, fine-tuning and evaluation debt: Write a customisation decision note with the target behaviour, a simpler baseline, training-data requirements, two evaluation criteria and a maintenance owner.

Combine the five outputs in the downloadable course workbook. Write a short decision note explaining the problem, your chosen approach, the evidence used and one limitation that would matter in real use. Keep initial and revised versions where a check changed your answer.

Review against these criteria

  • Map where data goes in a supposedly local setup.
  • Estimate weight storage without confusing it with total memory.
  • Explain why longer conversations can increase inference memory.
  • Measure the performance that the task actually needs.
  • Plan what must be maintained after customisation.

Ask a colleague or learning partner to challenge one assumption if available. If working alone, use the worked examples to review your reasoning and record the limits of that self-review. Completion is a personal learning record, not independent certification.

Change one condition

Choose one input or assumption from your original exercise and change it. Explain which part of the result must be recalculated, rechecked or withheld. Keep the revised answer and a short explanation of why the change matters. This is a transfer exercise: repeating the model answer without adapting it is insufficient.

Continue practising

Try a related Academy workspace or choose another course. The existing Practice Lab portfolio saves simulation work; keep this course workbook with your own learning records. Course completion progress remains in your browser and can be backed up through My Library.