Documentation / ML Studio
LLM Studio and RAG
Specialize local language models with SFT, DPO, ORPO, KTO, QLoRA and more, quantize them behind a quality gate, serve them locally and answer from your documents.
On this page
LLM Studio (the Language tab) specializes small language models with your examples. Knowledge indexes your documents so a model can answer from them (RAG). Both work offline, on your computer.
Local base models
A base model is an already trained model that you are going to adapt. In Local base models you see the ones
in the project's models/base folder. They are only loaded in safetensors format, never with remote code.
| Base | Typical use |
|---|---|
| SmolLM2, Qwen2.5, Phi-3.5 | Generation and chat. |
| Multilingual BERT | Classification and entity extraction. |
| E5, MiniLM | Embeddings for search and RAG. |
Downloading a base from the catalog requires the project to allow network access (policies.allowNetwork: true)
and your approval. Create tiny base creates a very small model that trains offline on any CPU, to try out the
flow.
models/
└── base/
└── qwen2.5-0.5b-instruct/
├── config.json
├── model.safetensors
└── tokenizer.jsonFine-tuning methods
| Method | Data | What for |
|---|---|---|
| SFT (LoRA) | JSONL with messages or prompt/completion | Teach it to answer with examples. LoRA trains only a small adapter. |
| DPO | prompt, chosen, rejected | Align with preferences (good answer versus bad answer). |
| ORPO | prompt, chosen, rejected | Preferences without a reference model. |
| KTO | prompt, completion, label | Unpaired preferences. |
| IPO | Preference pairs | A DPO variant that is more robust to overfitting. |
| RLHF | Human preferences | Reinforcement learning from human feedback. |
| RLAIF | Preferences from a judge model | Reinforcement learning from AI feedback. |
| Distillation | Answers from a teacher model | Transfer what a large model knows to a small one. |
| QLoRA | Same as SFT | Base loaded in 4 bits to train larger models with little VRAM. |
| Text classifier | CSV with text and label | An encoder that classifies texts. |
An example of SFT data:
{"messages": [{"role": "user", "content": "How do I reset my password?"},
{"role": "assistant", "content": "Go to Account › Security and press “Change password”."}]}QLoRA only works with NVIDIA GPUs and CUDA. Fine-tuning a small LLM with LoRA is possible on a CPU, but slow. See Deep learning and GPU.
As with tabular models, there are live curves, pause and resume. When it finishes you can Merge adapter to get a single model.
Evaluation and guardrails
When a generative fine-tune finishes, Yuli queues an evaluation on its own: perplexity (lower is better), exact match and token-level F1 over a set of questions, contamination (whether the questions were in the training data) and speed (Tokens/s, p95 latency).
The manifest's guardrails (llm.guardrails) block patterns and mask personal data in every generation, both in
the Try panel and in the service.
Quantization
Quantize stores the weights with fewer bits (int8, int4 or GGUF) so the model takes up less space and runs faster. Yuli applies a quality gate: it compares the quantized model with the original on data it hasn't seen and never accepts a version that loses quality beyond the tolerance. If no option passes, the job fails with the numbers and no worse version is created.
Built-in inference engine
Local service starts Yuli's own server on 127.0.0.1, compatible with the OpenAI API. Yuli's inference engine
loads safetensors, LoRA adapters, quantized versions, GGUF and ONNX without installing anything else.
import os
from openai import OpenAI
# You only change the base URL. The key, if you use one, comes from an environment variable, never from the code.
client = OpenAI(base_url="http://127.0.0.1:<port>/v1", api_key=os.environ["LOCAL_LLM_KEY"])
reply = client.chat.completions.create(
model="soporte-es",
messages=[{"role": "user", "content": "Where can I see my license?"}],
)
print(reply.choices[0].message.content)Knowledge (RAG)
In Knowledge, Create knowledge base indexes a project folder (with globs to choose the files) into chunks. It uses Yuli's own embeddings, offline, or a local model if you specify one.
| Search | How it finds |
|---|---|
| Hybrid | Combines meaning and words. Usually the best option. |
| Vectors | By meaning (embeddings). |
| Words (BM25) | By term matching. |
The search explains why each chunk ranked. Retrieval quality measures, over a set of questions, whether the right chunk appears among the first results (recall@k, MRR).
Measure retrieval before blaming the model: if the search doesn't find the chunk, no LLM can answer well.
Next: Evaluation and models.