Evaluate model performance across multiple dimensions on QwenCloud, with LLM-based scoring and manual annotation.
Model evaluation quantifies model output quality across multiple dimensions. You can create evaluation dimensions and tasks on the Model Evaluation console page.
Model evaluation consists of two sub-modules:
Model evaluation costs consist of two parts:
Core concepts
Model evaluation consists of two sub-modules:
- Evaluation dimensions — Define the scoring criteria. Supports LLM-based scoring (numeric / classification) and manual annotation.
- Evaluation tasks — Select dimensions + dataset + model to evaluate, then view results.
Evaluation workflow
1
Define evaluation dimensions
Create one or more evaluation dimensions, choosing the dimension type, judge model, and scoring template. See Evaluation dimensions.
2
Prepare evaluation data
Create and publish an evaluation dataset in Datasets.
3
Create an evaluation task
Select the data source, evaluation model, and evaluation dimensions, then submit the task. See Evaluation tasks.
4
View results
Check scores, token usage, and per-sample scoring details in the task list.
Billing
Model evaluation costs consist of two parts:
| Cost type | Billed? | Description |
|---|---|---|
| Evaluated model inference | Yes | When using an "Evaluation dataset" as the data source, the system calls the evaluated model to run inference, billed at standard token rates. No inference cost when using "Inference results". |
| Judge model scoring | Free (limited time) | The judge model scoring process for LLM-based dimensions (numeric and classification) is currently free for a limited time. |
Next steps
- Evaluation dimensions -- Learn about the three dimension types and create custom scoring criteria.
- Evaluation tasks -- Create evaluation tasks and view results.
- Create a dataset -- Prepare an evaluation dataset.