Create and manage evaluation dimensions on QwenCloud with LLM numeric scoring, LLM classification, and manual annotation.
Evaluation dimensions define the scoring criteria for evaluation tasks. You can create and manage dimensions on the Evaluation Dimensions console page.
A single evaluation task can combine multiple dimensions to assess model performance from different perspectives.
For scenarios requiring quantitative scoring (e.g. response quality rated 1-5).
For binary pass/fail scenarios (e.g. whether a response meets specific criteria, content safety checks).
For scenarios requiring human judgment. No judge model or prompt configuration is needed.
LLM-based dimensions support the following judge models (subject to console availability):
On the evaluation dimensions list page, you can:
Dimension types
| Type | Description | Scoring method | Use cases |
|---|---|---|---|
| LLM - Numeric | A judge model scores responses on a numeric scale (e.g. 0-5) | Numeric output | Q&A quality, content generation quality |
| LLM - Classification | A judge model classifies responses as pass/fail | Label output | Content safety, correctness checks |
| Manual - Classification | Human annotators label responses using custom categories | Manual labeling | Creativity evaluation, expert judgment |
Create an evaluation dimension
1
Open the create page
On the evaluation dimensions list page, click Create.
2
Enter basic information
Enter a dimension name (required) and description (optional, used to describe the evaluation purpose of the dimension).
3
Select the dimension type
Choose LLM - Numeric, LLM - Classification, or Manual - Classification. The remaining fields vary by type.
4
Configure scoring rules
Complete the type-specific configuration (see below), then click Create.
LLM - Numeric
For scenarios requiring quantitative scoring (e.g. response quality rated 1-5).
| Field | Description |
|---|---|
| Judge model | The AI model that performs scoring |
| Scoring template | A preset scoring prompt template. Options: General evaluation (default — scores across relevance, informativeness, clarity, etc.), Semantic similarity (evaluates how closely the output matches the reference answer), Custom scorer (write your own scoring prompt) |
| Prompt | Scoring instructions. Supports variables ${prompt}, ${output}, and ${completion} |
| Score range | Minimum and maximum scores (default 0-5) |
| Pass threshold | Scores at or above this value are considered passing (default 3) |
The following variables can be used in prompts and are automatically replaced with actual content during evaluation:
${prompt}— User input (the question from the evaluation data)${output}— Model output (the response generated by the evaluated model)${completion}— Reference answer (the expected answer from the evaluation data, used when a standard answer is available)
LLM - Classification
For binary pass/fail scenarios (e.g. whether a response meets specific criteria, content safety checks).
| Field | Description |
|---|---|
| Judge model | The AI model that performs classification |
| Scoring template | A preset classification prompt template. Options: Standard matching (default), Sentiment analysis (classifies response sentiment), Custom scorer (write your own classification prompt) |
| Prompt | Classification instructions. Supports variables ${prompt}, ${output}, and ${completion}. The judge model outputs [[Pass]] or [[Fail]] |
| Labels | Define pass and fail label sets. Custom labels can be added (up to 20 per category) |
Manual - Classification
For scenarios requiring human judgment. No judge model or prompt configuration is needed.
| Field | Description |
|---|---|
| Labels | Define pass and fail label sets. Custom labels can be added (up to 20 per category) |
Judge models
LLM-based dimensions support the following judge models (subject to console availability):
- Qwen3.7-Plus-2026-05-26
- Qwen3.7-Max-2026-06-08
- Qwen3.6-Flash-2026-04-16
- DeepSeek-V4-Pro
- DeepSeek-V4-Flash
Manage evaluation dimensions
On the evaluation dimensions list page, you can:
- Filter — Use the type dropdown to filter by dimension type.
- Details — Click the details button in the actions column to view dimension settings.
- Delete — Click the delete button in the actions column to remove a dimension.
Evaluation dimensions referenced by evaluation tasks cannot be deleted. To delete a dimension, first delete the evaluation tasks that reference it.
Next steps
- Evaluation tasks -- Use evaluation dimensions to create evaluation tasks.
- Create a dataset -- Prepare an evaluation dataset.