已合并
docs: add loss_ctx parameter docstring for DeepseekV4 and MiniMaxM2 CausalLM #4930
Rostellaria创建于 20 天前
docs: add loss_ctx parameter docstring for DeepseekV4 and MiniMaxM2 CausalLM #4930
已合并
共 2 个文件变更+8-0
| @@ -1701,6 +1701,10 @@ class DeepseekV4ForCausalLM(DeepseekV4PreTrainedModel, GenerationMixin): | |||
| 1701 | Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., | 1701 | Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., |
| 1702 | config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored | 1702 | config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored |
| 1703 | (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. | 1703 | (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. |
| 1704 | + loss_ctx (`callable`, *optional*): | ||
| 1705 | + A callable with signature `(hidden_states, weight, bias) -> loss` that lets the model compute the | ||
| 1706 | + loss directly without materialising the full logits. When provided, `labels` is ignored for the main loss | ||
| 1707 | + path and the returned `logits` are `None`. Note that `labels` is still used for the MTP loss path. | ||
| 1704 | 1708 | ||
| 1705 | Example: | 1709 | Example: |
| 1706 | 1710 | ||
| @@ -820,6 +820,10 @@ class MiniMaxM2ForCausalLM(MiniMaxM2PreTrainedModel, GenerationMixin): | |||
| 820 | Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., | 820 | Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., |
| 821 | config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored | 821 | config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored |
| 822 | (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. | 822 | (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. |
| 823 | + loss_ctx (`callable`, *optional*): | ||
| 824 | + A callable with signature `(hidden_states, weight, bias) -> loss` that lets the model compute the | ||
| 825 | + loss directly without materialising the full logits. When provided, `labels` is ignored for the main loss | ||
| 826 | + path and the returned `logits` are `None`. | ||
| 823 | 827 | ||
| 824 | Example: | 828 | Example: |
| 825 | 829 | ||