Reference for all fine-tuning hyperparameters available in Expert mode on Qwen Cloud.
Hyperparameters control how the model learns during fine-tuning. In Simple mode, Qwen Cloud applies optimized defaults. In Expert mode, you can customize all parameters in the console or via API.
The parameters
Hyperparameters table
| Parameter | Default | Range | Description |
|---|---|---|---|
batch_size | 16 | 8 -- 1024 | Number of training examples processed in each step. Larger values improve stability but require more memory. |
learning_rate | 0.0001 | > 0 | Controls the magnitude of weight updates. Too high causes instability; too low slows convergence. |
n_epochs | 3 | 1 -- 200 | Number of complete passes through the training dataset. More epochs increase training time and cost. |
eval_steps | 50 | 1 -- 2147483647 | Number of training steps between validation evaluations. Lower values give more frequent metrics but slow training. |
lr_scheduler_type | Linear | Linear, Cosine, Inverse Sqrt | Strategy for adjusting learning rate during training. |
max_length | 32768 | 500 -- 32768 | Maximum token length for a single training example. Examples exceeding this length are discarded. |
warmup_ratio | 0.1 | 0 -- 1 | Proportion of total training steps used for learning rate warmup. Warmup helps stabilize early training. |
weight_decay | 0.01 | 0 -- 0.2 | L2 regularization strength. Helps prevent overfitting and maintains generalization. |
lora_rank | 64 | 8, 16, 32, 64, 128 | Rank of the LoRA adapter matrices. Higher values increase capacity but also compute cost. |
lora_alpha | 16 | 8, 16, 32, 64 | Scaling factor for LoRA updates. Controls the influence of the adapter relative to the base model. |
lora_dropout | 0.05 | 0 -- 0.2 | Dropout rate applied to LoRA layers. Helps prevent overfitting in the adapter weights. |
LoRA-specific parameters
The parameters lora_rank, lora_alpha, and lora_dropout are specific to the LoRA training method used by all fine-tuning jobs.
Tips for tuning
- Start with defaults. The default values work well for most use cases. Only adjust parameters if your initial results are unsatisfactory.
- Adjust learning rate carefully. If training loss does not decrease, try increasing the learning rate. If loss is unstable or spikes, reduce it.
- Monitor validation metrics. Use the Metrics tab on the job detail page to watch for overfitting -- when validation loss increases while training loss continues to decrease.
- Increase epochs for small datasets. With fewer training examples, more epochs give the model additional passes to learn patterns.
- Use warmup for large learning rates. Setting
warmup_ratioto 0.05 -- 0.1 helps avoid early training instability.