Documentation / ML Studio

LLM Studio and RAG

Specialize local language models with SFT, DPO, ORPO, KTO, QLoRA and more, quantize them behind a quality gate, serve them locally and answer from your documents.

On this page

LLM Studio (the Language tab) specializes small language models with your examples. Knowledge indexes your documents so a model can answer from them (RAG). Both work offline, on your computer.

Local base models

A base model is an already trained model that you are going to adapt. In Local base models you see the ones in the project's models/base folder. They are only loaded in safetensors format, never with remote code.

BaseTypical use
SmolLM2, Qwen2.5, Phi-3.5Generation and chat.
Multilingual BERTClassification and entity extraction.
E5, MiniLMEmbeddings for search and RAG.

Downloading a base from the catalog requires the project to allow network access (policies.allowNetwork: true) and your approval. Create tiny base creates a very small model that trains offline on any CPU, to try out the flow.

text
models/
└── base/
    └── qwen2.5-0.5b-instruct/
        ├── config.json
        ├── model.safetensors
        └── tokenizer.json

Fine-tuning methods

MethodDataWhat for
SFT (LoRA)JSONL with messages or prompt/completionTeach it to answer with examples. LoRA trains only a small adapter.
DPOprompt, chosen, rejectedAlign with preferences (good answer versus bad answer).
ORPOprompt, chosen, rejectedPreferences without a reference model.
KTOprompt, completion, labelUnpaired preferences.
IPOPreference pairsA DPO variant that is more robust to overfitting.
RLHFHuman preferencesReinforcement learning from human feedback.
RLAIFPreferences from a judge modelReinforcement learning from AI feedback.
DistillationAnswers from a teacher modelTransfer what a large model knows to a small one.
QLoRASame as SFTBase loaded in 4 bits to train larger models with little VRAM.
Text classifierCSV with text and labelAn encoder that classifies texts.

An example of SFT data:

json
{"messages": [{"role": "user", "content": "How do I reset my password?"},
              {"role": "assistant", "content": "Go to Account › Security and press “Change password”."}]}

QLoRA only works with NVIDIA GPUs and CUDA. Fine-tuning a small LLM with LoRA is possible on a CPU, but slow. See Deep learning and GPU.

As with tabular models, there are live curves, pause and resume. When it finishes you can Merge adapter to get a single model.

Evaluation and guardrails

When a generative fine-tune finishes, Yuli queues an evaluation on its own: perplexity (lower is better), exact match and token-level F1 over a set of questions, contamination (whether the questions were in the training data) and speed (Tokens/s, p95 latency).

The manifest's guardrails (llm.guardrails) block patterns and mask personal data in every generation, both in the Try panel and in the service.

Quantization

Quantize stores the weights with fewer bits (int8, int4 or GGUF) so the model takes up less space and runs faster. Yuli applies a quality gate: it compares the quantized model with the original on data it hasn't seen and never accepts a version that loses quality beyond the tolerance. If no option passes, the job fails with the numbers and no worse version is created.

Built-in inference engine

Local service starts Yuli's own server on 127.0.0.1, compatible with the OpenAI API. Yuli's inference engine loads safetensors, LoRA adapters, quantized versions, GGUF and ONNX without installing anything else.

python
import os
from openai import OpenAI

# You only change the base URL. The key, if you use one, comes from an environment variable, never from the code.
client = OpenAI(base_url="http://127.0.0.1:<port>/v1", api_key=os.environ["LOCAL_LLM_KEY"])
reply = client.chat.completions.create(
    model="soporte-es",
    messages=[{"role": "user", "content": "Where can I see my license?"}],
)
print(reply.choices[0].message.content)

Knowledge (RAG)

In Knowledge, Create knowledge base indexes a project folder (with globs to choose the files) into chunks. It uses Yuli's own embeddings, offline, or a local model if you specify one.

SearchHow it finds
HybridCombines meaning and words. Usually the best option.
VectorsBy meaning (embeddings).
Words (BM25)By term matching.

The search explains why each chunk ranked. Retrieval quality measures, over a set of questions, whether the right chunk appears among the first results (recall@k, MRR).

0 of 4 done

Measure retrieval before blaming the model: if the search doesn't find the chunk, no LLM can answer well.

Next: Evaluation and models.