main

GGitHubStop discovering tiny models from trl-internal-testing (#4062)

932c36c0创建于 5 天前历史提交

文件	最后提交记录	最后更新时间
backend	[fix] Save the external data of an ONNX model over 2GB when exporting (#3913) * Save the files that hold the weights of an ONNX model over 2GB alongside it An ONNX model larger than the 2GB protobuf limit keeps its weights in separate files and records their locations inside itself, but only the ONNX file was saved, leaving a model that cannot be loaded. The locations are read from the model rather than derived from its name, since they differ between the exporter and the optimizer. * Name the external data file of an exported ONNX model the way Optimum fetches it ONNX Runtime writes the weights of an optimized model to <model>.onnx.data, but Optimum only ever requests <model>.onnx_data when loading, and swallows the resulting error. A model pushed to the Hub therefore loads without its weights. Renaming the file, and the location recorded in the model, makes the upload usable. * Prepare the external data of an exported ONNX model in a single pass The two helpers each loaded the ONNX file and built the same list of location entries, so a model whose weights sit inline was parsed twice to learn that it has no external data. One function does both jobs off one parse, which halves that cost (1.48s to 0.76s on a 1.46 GB model). Optimum is the only caller that reaches this, and it writes at most one file under a plain name beside the model, so the guards for an absolute location, a location escaping the model directory, and a location with no file behind it were all unreachable. The docstring records the assumption instead. * Drop a semicolon from the save_model docstring --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	26 天前
base	Stop discovering tiny models from trl-internal-testing (#4062) * Drop trl-internal-testing from the tiny model discovery authors * Repoint the tiny models that trl-internal-testing is retiring	5 天前
cross_encoder	Stabilize ListMLE loss for large finite scores (#4012) * Stabilize ListMLE loss for large finite scores * Trim ListMLE comments and redundant gradient assertion --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	11 天前
multi_vector_encoder	Optionally use fastcluster for hierarchical token pooling linkage (#3941) * feat: use fastcluster for hierarchical token pooling linkage * Make fastcluster an optional backend for hierarchical token pooling Reworks #3941. fastcluster produces the same Ward clustering as scipy (moderately faster, and it releases the GIL), but as a compiled package it is too heavy for a mandatory dependency. - Resolve the linkage backend once per process. Python does not cache failed imports, so resolving per document would walk sys.path on every call when fastcluster is absent. - Extract the condensed matrix with squareform(checks=False) instead of a triu_indices gather (bitwise identical, roughly 2-8x faster). - Move fastcluster to the dev extras and the test dependency group so CI exercises the fast path while the scipy fallback keeps its test. - Add a fallback parity test. --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	23 天前
sentence_transformer	Group tied scores when evaluating paraphrase mining (#4028) * Group tied scores when evaluating paraphrase mining Update precision, recall and F1 only after each complete score group, and weight its average-precision contribution by all positives in that group. * Cover multiple positives in tied paraphrase scores --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	10 天前
sparse_encoder	[`fix`] Support disabling CSR auxiliary latents with k_aux=0 (#4017) * fix: skip aux decoding when k_aux=0 * fix: skip L_aux when no aux representation * Add regression coverage * Simplify CSR auxiliary loss handling and verify gradients --------- Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	11 天前
util	Reuse calibration ranges across multi-vector embeddings (#4025) * Reuse calibration ranges across multi-vector embeddings Signed-off-by: Ayoub Chikri <ayoub.chikri@student-cs.fr> * Simplify quantization comments --------- Signed-off-by: Ayoub Chikri <ayoub.chikri@student-cs.fr> Co-authored-by: Tom Aarsen <Cubiegamedev@gmail.com>	10 天前
__init__.py	[`ci`] Attempt to fix CI disk space issues (#2906) * Also clear cache mid-tests for Windows after CI runner update * Split test_pretrained tests into individual tests This allows us to more easily clear the cache after every single model. * Attempt to delete unnecessary files from the runner image on Linux * Also print free disk space prior to clearing space * Try to also clear space for Windows * Remove /q flag * Also remove the /s flag * RMDIR -> Remove-Item * Add ignore_cleanup_errors=True to TempDir to avoid NotADirectoryError on Windows * Replace ignore_cleanup_errors with a manual SafeTemporaryDirectory * Add ignore_cleanup_errors=True when possible * Increase versions of GH actions	2 年前
conftest.py	Tests: Use a new cached tiny MVE model (#4024)	11 天前
utils.py	Distribute evaluator inference across DDP ranks (#4004) * Run the evaluator once under DDP and broadcast its metrics evaluation_loop() called self.evaluator(...) on every rank, only suppressing logging on non-main ranks. Under a DistributedSampler each rank evaluates a different shard, so the reported metrics differ per rank for the same step. Only world process zero now runs the evaluator; broadcast_object_list() sends its metrics dict to the other ranks so output.metrics stays populated everywhere, the same shape #3463 reverted the local_process_zero-skip version of this fix over. Fixes #3556 * Distribute evaluator inference across existing training ranks Share local inference between direct calls, multiprocessing pools, and distributed evaluation. Resolve options once and combine outputs before final conversion. * Fix distributed evaluation with older PyTorch inference mode * Preserve empty prediction device and refine inference return types * Centralize evaluator rank routing and preserve sharded backend participation --------- Co-authored-by: Amir Fathi <amirfathi.me@gmail.com>	11 天前