Documentation / ML Studio
Data in ML Studio
Import and profile datasets, immutable versions, preparation, quality, sensitivity with masking, and lineage.
On this page
Almost every bad model starts with a problem that was already in the data. The Data tab helps you see it before training. A dataset is a table: rows (examples) and columns (variables and the column to predict).
Import and profile
Press Import and profile and choose a file or a database.
| Source | Detail |
|---|---|
| Files | CSV, TSV, JSON, JSONL, Parquet and SQLite. |
| Database | A read-only SELECT query on a Database connection. |
| Large data | Works with datasets larger than RAM. |
When importing, Yuli computes the full profile and shows you the Masked sample: the example rows with the sensitive columns hidden.
Immutable versions
Every change (import, clean, split) creates a new, immutable version, identified by a content hash. A version is never overwritten: a model can always say exactly which data it was trained on.
ventas.csv
├── v1 sha256:4f1c… imported
├── v2 sha256:9a07… remove duplicates
├── v3 sha256:c2e8… fill nulls (median)
└── v4 sha256:71bd… split 70 / 15 / 15The views of a dataset
| View | What it shows |
|---|---|
| Profile | Schema and per-column statistics. |
| Prepare | Cleaning and transformation steps. |
| Card | The dataset card, with its license (required to approve a model). |
| Data | The rows, masked where appropriate. |
| Sensitivity | Which columns are sensitive and how they are masked. |
| Quality | An overall score and its dimensions. |
| Lineage | Where each version comes from and which models use it. |
Profile
The column table shows Column, Type, Nulls, Unique, Distribution and Summary. Below it:
- Class balance: how many examples there are of each value of the column to predict.
- Strongest correlations: the pairs of columns that move together the most.
- Detected issues: constant columns, identifiers worth excluding, many nulls, duplicate rows or imbalanced classes.
A near-perfect correlation with the column to predict is usually a leak: information from the answer that slips into the variables. The model will look excellent and fail in production. Exclude that column.
Prepare the data
Each Prepare step produces a new version:
| Step | What it does |
|---|---|
| Remove duplicates | Deletes repeated rows. |
| Fill nulls | Fills in missing values. |
| One-hot | Turns a category into columns of 0 and 1. |
| Standardize | Centers on 0 with a standard deviation of 1. |
| Normalize (0–1) | Scales to the range 0 to 1. |
| Split | Separates into training, validation and test. |
Quality
The Quality view gives an Overall score and a Grade, and breaks it down into dimensions:
| Dimension | Question |
|---|---|
| Missing | How many values are missing, and where? |
| Validity | Do the values have the expected type and format? |
| Duplicates and rules | Are there repeated rows or broken business rules? |
| Outliers | Are there suspicious extreme values? |
| Distributions | Do the columns have reasonable shapes? |
| Time and drift | Does the data change over time? |
| Labels | Is the column to predict consistent? |
| Bias | Is any group under- or over-represented? |
Sensitivity and masking
In Sensitivity, each column has a probability of being sensitive, a category and a suggested masking. The detector combines the column name, patterns and check-digit validators (cards, IBAN, CLABE, CPF, CURP/RFC).
- You and your local models work with the real values: they don't leave your computer.
- External agents only see sensitive columns as
•••. Aggregates do get through: an agent can reason about which variables matter most without seeing a single customer. - Only a person can mark a column as not sensitive. An agent can never unmask data for itself.
Sensitive columns are recorded in yuli-ml.yaml:
sensitiveColumns:
- email
- curp
- telefonoIf the model affects people, also review the Bias dimension in Quality and the Robustness and fairness section of the evaluation. See ML privacy and sandbox.
Lineage
Lineage draws the full chain: source file or query → dataset versions → preparation steps → training runs →
model versions. Each model keeps its lineage.json, so you can always answer “exactly which data was the version in
production trained on?”.
Next step: Training.