Private AI gets sold with a lot of vocabulary. This page explains the terms I use when I size a system, what the hardware actually does, how retrieval and agents fit together, and what I set up with all of it for your office: private RAG over your documents, agents that do real work, and automations with a person in the loop.
Every item on this page is something I install, connect and support. Here is what each one looks like in a real engagement. The glossary below explains the terms.
I index your approved sources, such as file shares, SharePoint, SOPs, contracts and tickets, into a local vector database. The model answers from them with the source cited, and nothing leaves your network. This is the core of every appliance I install.
Using MCP and direct integrations, the AI can read from M365, your CRM, your accounting system or a SQL database. Read only by default, every query logged, limited to what each person is already allowed to see.
One workflow at a time. Contract intake, document chasing, request triage, report assembly. The agent uses tools to do the steps, a person approves anything that matters, and we measure the hours saved.
A VLM can read scanned paperwork, describe what a camera sees, or flag an image for review. Useful when the data you care about is not text.
If you already pay for Copilot, ChatGPT or Claude, I set it up correctly, lock down the data settings, shut off unapproved tools and report monthly on usage and cost. Sensitive work stays on the appliance.
If you understand these twelve, you can read any model spec sheet and know what it means for your office.
Large Language Model. An AI model trained to understand and generate language, code and other text.
Running AI models on your own computer or server instead of sending every request to a cloud provider.
Running an already trained model to produce answers or predictions. This is what your appliance does all day.
The numerical values inside a model that determine how it processes information. A 70B model has about 70 billion of them.
The learned values of a model's parameters. Downloading a pretrained model mostly means downloading its weights.
The small units of text a model reads and writes. A token might be a word, part of a word or a piece of punctuation.
How much a model can consider in one request, including the conversation so far and any documents you hand it.
Memory on a graphics card. It is the single biggest factor in which models you can run and how fast they answer.
Shrinking the precision of model weights so they fit in less memory, at a small cost in accuracy.
A common model file format built for efficient local inference, especially with llama.cpp based software.
How quickly a model generates output. Higher is faster. This is the number you feel when you use it.
Memory that stores attention data from tokens already processed, so the model does not redo that work for each new word.
These are the pieces that turn a chatbot into something that knows your files and can do work on your behalf.
Retrieval Augmented Generation. The model looks up relevant passages in your documents or databases before it answers, and cites them. This is how a private appliance knows your files without being trained on them.
Numerical representations of text that capture meaning. They are what make semantic search and document retrieval possible.
Stores and searches embeddings. It is the index behind a RAG system, and on an Airman AI build it lives on your network.
Additional training that adapts a model's behavior, style or capabilities for specific tasks.
A lightweight way to fine tune a model without rewriting all of its original weights. Cheaper and faster than full training.
Vision Language Model. A model that can look at images and answer questions about them. Useful for camera feeds, scanned documents and image based automation.
Lets a model ask external software, APIs or databases to do something, instead of only producing text.
Systems that use a model to complete multistep tasks, usually with tools, decisions and feedback along the way. A person approves anything that matters.
Model Context Protocol. A standard for connecting AI applications to tools and data sources, so one connector works across many AI products.
Running a model as a service so other applications and users can reach it through an API, with access controls and logging.
Every Airman AI system is sized to your team on these three questions.
The GPU does most of the math while VRAM holds the model weights and working data. Capacity decides which models fit. Processing speed decides how fast they answer. Both matter.
Several GPUs can share a model that will not fit on one card. Memory is not simply added together though, and the traffic between cards can slow things down. Sizing this right is most of the job.
A dedicated machine that hosts models for many users or applications. The questions that matter are simultaneous requests, latency, power draw, security and uptime.
Illustrative. Retrieval and tool execution can happen at different stages, and every external system sits behind its own access controls.
Bring a few documents your team searches every week. I'll show you RAG answering from them, live, on a box that never talks to the cloud.