Skip to main content

Training

The training key of the config.

Controls the model training process.

FieldTypeRequiredDefaultDescription
max_epochsinteger (≥ 1)yesThe maximum number of epochs to train for.
batch_sizeinteger (≥ 1)yesThe number of inputs to process in each training batch (see: input_preprocessing).
optimizerOptimizeryesControls the optimization algorithm used to update the model's weights.
schedulerScheduleryesControls how the learning rate is modified over the course of training.
encoder_training_strategyEncoderTrainingStrategyyesControls how encoder weights are managed during training.
early_stoppingEarlyStoppingnonullEnd fine tuning early if model performance is no longer improving.

Optimizer

One of the following, selected by its name field:

nameType
adamwAdamW
manualEscapeHatchComponent

Scheduler

One of the following, selected by its name field:

nameType
manualEscapeHatchComponent
plateauPlateau

EncoderTrainingStrategy

One of the following, selected by its name field:

nameType
freeze_unfreezeEncoderFreezing
layer_wise_lr_decayLayerWiseLRDecay

EarlyStopping

Controls early-stopping behavior during supervised fine tuning.

Monitors a validation metric and stops training after patience epochs without improvement. When monitoring begins depends on the encoder strategy: immediately for layer-wise LR decay, or after the encoder unfreeze epoch for freeze/unfreeze.

FieldTypeRequiredDefaultDescription
patienceinteger (≥ 1)yesEpochs without improvement before stopping.

AdamW

FieldTypeRequiredDefaultDescription
name"adamw"yes
lrnumber (≥ 0)yesThe learning rate to use for the optimizer.

Plateau

Lowers learning rate via an exponential decay strategy when model performance plateaus.

FieldTypeRequiredDefaultDescription
name"plateau"yes
factornumber (≥ 0, ≤ 1)yesMultiplicative factor of learning rate decay. LR = LR * factor.
patienceinteger (≥ 0)yesNumber of epochs with no improvement after which learning rate will be reduced.
min_lrnumber (≥ 0)yesA lower bound on the learning rate.
cooldowninteger (≥ 0)yesNumber of epochs to wait before resuming normal operation after lr has been reduced.

EncoderFreezing

Freeze encoder weights for a number of epochs, then unfreeze at a reduced learning rate.

FieldTypeRequiredDefaultDescription
name"freeze_unfreeze"no"freeze_unfreeze"
unfreeze_at_epochinteger (≥ 0)yesThe epoch at which weights in the encoder can be updated by the optimizer.
unfreeze_lr_factornumber (≥ 0)yesThe factor by which the learning rate is multiplied when the encoder is unfrozen.

LayerWiseLRDecay

Apply per-layer learning rate decay to the encoder from the start of training.

Each encoder layer receives a learning rate scaled by decay_factor relative to the next layer. The configured optimizer learning rate applies to the decoder (last layer). Layer i gets: lr * decay_factor^(num_layers - i).

FieldTypeRequiredDefaultDescription
name"layer_wise_lr_decay"yes
decay_factornumber (> 0, ≤ 1)yesMultiplicative factor applied per layer. Layer i LR = base_lr * decay_factor^(num_layers - i).
num_layersinteger (≥ 1)yesNumber of encoder layers. Must match the foundation model metadata (e.g. 4 for Nano, 12 for Tiny/Base).