| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
ggml-virtgpu: Fix some build commands (#20341) | 6 个月前 | |
common: migrate the deprecated --mmap/--no-mmap to --load-mode (#26934) Replace the deprecated --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, examples, and documentation. Internal warning message and env var docs updated accordingly. Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org> | 1 个月前 | |
make : deprecate (#10514) * make : deprecate ggml-ci * ci : disable Makefile builds ggml-ci * docs : remove make references [no ci] * ci : disable swift build ggml-ci * docs : remove obsolete make references, scripts, examples ggml-ci * basic fix for compare-commits.sh * update build.md * more build.md updates * more build.md updates * more build.md updates * Update Makefile Co-authored-by: Diego Devesa <slarengh@gmail.com> --------- Co-authored-by: slaren <slarengh@gmail.com> | 1 年前 | |
CANN: update docker images to 8.5.0 and improve CANN.md (#20801) * cann: update docker images to 8.5.0 - bump CANN base image from 8.3.rc2 to 8.5.0 - bump ASCEND_VERSION from 8.1.RC1.alpha001 to 8.5.0 Move to newer stable releases. * cann: update CANN.md * Update CANN.md to include BF16 support Added BF16 support information to the CANN documentation and corrected formatting for the installation instructions. * Fix formatting issues in CANN.md Fix 234: Trailing whitespace | 5 个月前 | |
docs : fix typos in CUDA-FEDORA.md and grammars/README.md (#24459) | 3 个月前 | |
ggml-et: Initial ET backend (#24179) * ggml-et: Add performance logging * ggml-et: Quants helpers * ggml-et: Add MUL_MAT kernel * ggml-et: Add ROPE kernel * ggml-et: Add RMS_NORM kernel * ggml-et: Add GLU kernel * ggml-et: Add SOFT_MAX kernel * ggml-et: Add GET_ROWS kernel * ggml-et: Add CONT kernel * ggml-et: Add SET_ROWS kernel * ggml-et: Add MUL_MAT_ID kernel * ggml-et: Build et kernels as part of ggml * ggml-et: Embed kernels with fs fallback * ggml-et: Build fixes * ggml-et: Add MUL_MAT F32xF32 op * ggml_et: Add MUL_MAT_ID op * ggml-et: Disable offloading for debug * ggml-et: Refactor out block ops * ggml-et: ggml backend API changes * ggml-et: Add RESHAPE/TRANSPOSE to supported * ggml-et: Add CONT_F16 * ggml-et: Add supported ops doc * gglm-et: Initial doc * ggml-et: Remove runtime import hacks We can now import the runtime by a simple find_package(), so we can cleanup the CMakeLists.txt. * ggml-et: Fix GET_ROWS kernel Fix lost batch dimension. Also clean vibe-comments. * ggml-et: Fix SET_ROWS kernel Remove incorrect broadcasting guard. * ggml-et: Use custom instruction for fp32->fp16 * ggml-et: Vectorize set_rows fp32->fp16 * ggml-et: Fix ROPE kernel (yarn) ggml-et: fix et_logf WIP: Fix ramp WIP: fix ROPE! * ggml-et: Better sinf * ggml-et: Fix SOFT_MAX Add max_bias and sink support. * ggml-et: Fix CONT Reorder from contiguous write to read with atomic stores. * ggml-et: Fix elmap kernel Remainder handlin * ggml-et: Fix MUL_MAT MUL_MAT_ID remainders * ggml-et: Fix ET-SOC reference * ggml-et: Fix embed kernels scripts for old python This allows GGML-ET to build on pre-3.8 python. * Add sysemu support with compile time flag -DGGML_ET_SYSEMU=ON (#6) * Example using ET-Soc-1 emulator configuration Example usage: bash cmake -B build -DGGML_CUDA=OFF -DGGML_ET=ON -DLLAMA_CURL=OFF -DGGML_CCACHE=ON cmake --build build --config Release -j $(nproc) time ./build/bin/test-backend-ops ./build/bin/llama-server \ --model Qwen3-0.6B-Q8_0.gguf \ --alias Qwen3-0.6B-Q8_0 \ -fa 0 \ --ctx-size 1024 \ --no-warmup \ --host 127.0.0.1 \ --port 8080 * build: proper dep tracking for kernels * support host using MOLD linker * initial multi core GET_ROW F32 implementation * vectorized q8 dequant * wip: cland warning clenaups and initial logging refactor * wip: message default message cleanup * chore: message cleanups * cmake cleanup * migrate to use platform provided functions * cmake back into subdir * support et_print() in kernels * fix: repair kernel building * perf: operations run async by default * debug: proper kernel dep tracking and error detection on kenrel launch * fix: kernel binary dep tracking and fixing get_rows_f32 erroring * perf: back to doing async kernel runs by default * perf: vectorize and parallel device memset * merge matmul work * misc: align allocation and enable all offload * misc: delete deadcode and respect memory limits * fix: repair tensor debug print * fix: loosen RMS_NORM op percision * feat: Q4_0 GET_ROWS * perf: FP32 MUL_MAT using TensorFMA * update limitations * perf: redue L1 load in compute_block_dot_product_q8_0 * feat: save kernel mapping (name to id) when profiling is enabled * chore: memops cleanup * perf: parallelize softmax by rows * perf: vectorize 2nd phase of softmax * perf: ban GET_ROWS from offloaded * perf: vectorize and non-atomic for eltwise ops and sub support * perf: vectorize normal rope * perf: glu runs in parallel * merge: manually merge saqib's work on kernel fixes * perf: more vectorized RoPE * perf: parallelize mul_mat_id * perf: parallelize set_rows_f32 * perf: vectorize softmax * feat: support kernel fusion and fuse RMS_NORM + MUL * fix: mostly resolve test-backend-ops failure in SOFT_MAX and ROPE * fix: bump max rope dims for gemma * feat: GeGLU and SCALE support to fully offload Gemma * perf: faster device memset * feat: get_rows supporting Q4_K and avoid cont cache coherent issues * better F32 MM * feat: NORM for ET backend * feat: SQR for ET backend * feat: UNARY on ET * feat: el_map support broadcasting for ET * feat: SUM_ROWS in ET backend * feat: more ops in ET backend * feat: WKV* operators in ET backend * perf: parallelize operators across cacheline instead of row * perf: parallelize get_rows on cacheline * wip: baseline FlashAttention for ET backend * wip: enough FA and CPY f32->f16 to run llama 3.1 fully offloaded with FA on * feat: f16 x f16 -> f32 MM using matrix engine * wip: f16 FlashAttention using matrix engine * wip: clean up * feat: barriers * perf: optimize FA_F16 in ET * perf: vectorize pack_k_for_transpose16 * perf: prefetch next loop matrix tile * perf: FlashAttention 2nd MM uses TensorFMA and optimizations * cleanup: flashattention reorg * perf: optimizations and fixes * feat: L2SCP API and make FlashAttention support DV = 256 for gemma * perf: parallelize norms beyond single row * feat: GATED_DELTA_NET support and relaxed L2_NORM requirment * feat: loosen RMS_NORM, NORM, ROPE contingous req too * feat: repeat supports brocasting on dim 0 and loosen cont check * feat: FILL and DIAG operator * feat: loosen UNARY support chcek * feat: TRI support * feat: SOLVE_TRI support * feat: basic SET support * feat: loosen CONT req * perf: fp16_to_fp32 use ASM * feat: IMROPE support * feat: PAD support * feat: global barrier * fix: view must live on the same backend as backing tensor * feat: relax CONCAT in ET backend * feat: dead simple CUMSUM implementation * feat: basic SSM_CONV support * feat: loosen CONCAT req * feat: relax GATED_DELTA_NET and add SET support proper * cleanup: cleanup LCM math * feat: SWIGLU single input * feat: SSM_SCAN support * feat: el_map supports non aligned tensors in best effort * feat: basic GROUP_NORM support * feat: loosen MUL_MAT capablities slightly * feat: loosen MUL_MAT and GET_ROWS and add IM2COL * feat: special case for softmax 1x1x1x1 * feat: loosen SOFT_MAX req in ET backend * fix: el_map unaligned acse fixes * perf: optimize zero_acc_vec in flash_attn_ext_f16_me * perf: use hart 1 for packing in MM and FA for FP16 * feat: kernel semaphore * perf: better instruction sequence in FlashAttention * fix: gated_delta_net with proper masking * perf: better parallelization for GATED_DELTA_NET * perf: parallelize SSM_CONV over nr * perf: vectorize SSM_CONV * perf: optimize MUL_MAT for q8 * feat: support Gemma 4 * fix: support multi-device * feat: broader GLU support * feat: unary ops supports view * fix: repair fp16 MM using matrix engine * perf: handle large N GEMV better * perf: better q8_0 MM * perf: better set_rows * add back deleted files * fix: repair after merge * feat: POC version of uberkernel * feat: RMS_NORM in uberkernel * feat: add more kernels into usage * chore: clean up uberkernel compilation * perf: faster flash attention * perf: opt flash attention for large seq length * feat: loosen op bounds. clamp and mean support * perf: vectorize ssm_scan * perf: slightly faster FA * perf: FlashAttention parallel MM and load * perf: fuse Q8 MM and ADD * feat: basic conv kernel for ET * softMAx_test * set_rows_f32 * get_rows and cont * testing * set_rows_exp * Junk addition * Narrowing the issue * Update flash_attn_ext_f16_me.c Focusing FA_ext_f16_me * test * Eviction updated * Detailed cache eviction debug * mulmat * removeal of BUILD_FOR_UBERKERNEL flag * cleaning... * fix: balance FCC0 count * feat: implement mul_mat and mul_mat_id for Q4_0 type * optimize uberkernel plan upload * add mul_mat q4 into uberkernel * enable gating flush to just uberkernel * update docs for ET * update op support for ET * et-backend: optimize Q4_0 and Q8_0 mul_mat_id row accumulations * et-backend: specialize mul_mat_id kernels for Q4_0 and Q8_0 * et-backend: fix RoPE YaRN corr_dim formula and handle degenerate inputs * test-backend-ops: add DeepSeek-V2-Lite RoPE test coverage * et-backend: add Q4_0 mul_mat matrix-engine kernel using TensorFMA32 * et-backend: vectorize Q4_0 matrix-engine dequantization * et-backend: support hybrid matrix/vector engine execution for Q4_0 mul_mat tail * et-backend: run partial-N tiles on matrix engine for Q4_0 mul_mat * et-backend: route Q4_0 mul_mat N < 53 to vecdot for better prefill latency * Update uberkernel.c * Update unary_f32.c * gemma 4 * bisect gemma4: enable scale_f32 only * bisect gemma4: +rms_norm_f32 * bisect gemma4: +rms_norm_mul_f32 * bisect gemma4: disable rms_norm_mul_f32 -- BREAKS OUTPUT * bisect gemma4: +rope_f32 (skip rms_norm_mul) * bisect gemma4: +el_map_f32 * bisect gemma4: +softmax_f32 * bisect gemma4: +get_rows_f32 * bisect gemma4: +glu_f32 * bisect gemma4: +mul_mat_f32 +mul_mat_f32_matrix_engine * bisect gemma4: +mul_mat_f16 +mul_mat_f16_matrix_engine * bisect gemma4: +mul_mat_Q8_0 +mul_mat_Q4_0 * bisect gemma4: +flash_attn_ext_f32 +flash_attn_ext_f16_me * bisect gemma4: +mul_mat_id_f32 * bisect gemma4: +sum_rows_f32 * bisect gemma4: +cont_f16 * bisect gemma4: +fill_f32 * bisect gemma4: +unary_f32 (all ops re-enabled except rms_norm_mul) * Update rms_norm_mul_f32.c * bisect2 gemma4 n64: +scale_f32 only * bisect2 gemma4 n64: +rms_norm_f32 +rope_f32 * bisect2 gemma4 n64: +rms_norm_mul_f32 (with ET_UBERKERNEL eviction fix) * bisect2 gemma4 n64: +el_map +get_rows +glu +softmax (skip rms_norm_mul) * bisect2 gemma4 n64: all ops enabled except rms_norm_mul * bisect2 n64: test unary+cont+fill+sum_rows (no mul_mat/flash_attn) * bisect2 n64: +mul_mat_f32 +mul_mat_f32_matrix_engine * bisect2 n64: +mul_mat_f16 +mul_mat_f16_matrix_engine * bisect2 n64: +mul_mat_Q8_0 +mul_mat_Q4_0 * bisect2 n64: +mul_mat_Q8_0 only (disable Q4_0) * bisect2 n64: +mul_mat_Q4_0 only (Q8_0 breaks) * bisect2 n64: +mul_mat_id +flash_attn_ext (skip Q8_0) * run-3: matmul + rms_norm_mul * run-4 * Revert "run-4" * run5 * changes after cleanup * cleanup before upstream * restrict changes into ET backend * move kernel embedding from Python to CMake * move uberkernel gen into CMake * apply clang format * update CMake style * update to match C and C++ style * use source ggml and quant headers instead of ET's * MROPE support * absorb view ops into same branch as none * fix bad rebase * add marty1885 to codeowners * oops * remove redundant newline * fix CI editor warnings --------- Co-authored-by: Vidas <vidas@nuolat.lt> Co-authored-by: Gianluca Guida <glguida@tlbflush.org> Co-authored-by: Gianluca Guida <gianluca@nekko.ai> Co-authored-by: ubergarm <leimgrub@gmail.com> Co-authored-by: SaqibAkram-10xE <saqib.akram@10xengineers.ai> Co-authored-by: Rehan Qasim <rehan.qasim@10xengineers.ai> | 2 个月前 | |
opencl: cache compiled cl_program binaries on disk (#26050) | 1 个月前 | |
ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292) * update to ov-2026.3, update device drivers * ci: skip nemotron-h rollback test on OpenVINO The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot preserve the recurrent cache output shape. Keep the test enabled for other backends and retain the qwen35 OpenVINO rollback coverage. --------- Co-authored-by: ravi9 <ravi.panchumarthy@intel.com> | 30 天前 | |
common: migrate the deprecated --mmap/--no-mmap to --load-mode (#26934) Replace the deprecated --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, examples, and documentation. Internal warning message and env var docs updated accordingly. Signed-off-by: Fathi Boudra <fathi.boudra@linaro.org> | 1 个月前 | |
ggml-virtgpu: improve the reliability of the code (#19846) * ggml-virtgpu-backend: validate the consistency of the received objects This patch adds consistency checks in the ggml-virtgpu-backend (running on the host side) to ensure that the data received from the guest is consistent (valid pointers, valid sizes and offsets). * ggml-virtgpu-backend: add fallback/skips for optional ggml backend methods 1. bck->iface.synchronize(bck) 2. buft->iface.get_alloc_size(buft, op) 3. buft->iface.get_max_size(buft) these three methods are optional in the GGML interface. get_max_size was already properly defaulted, but backend sychronize and butf get_max_size would have segfaulted the backend if not implemented. * ggml-virtgpu-backend: fix log format missing argument * ggml-virtgpu-backend: improve the abort message * ggml-virtgpu-backend: more safety checks * ggml-virtgpu-backend: new error code * ggml-virtgpu-backend: initialize all the error codes * ggml-virtgpu: add a missing comment generated by the code generator * ggml-virtgpu: add the '[virtgpu]' prefix to the device/buffer names * ggml-virtgpu: apir_device_buffer_from_ptr: improve the error message * ggml-virtgpu: shared: make it match the latest api_remoting.h of Virglrenderer APIR (still unmerged) * ggml-virtgpu: update the code generator to have dispatch_command_name in a host/guest shared file * ggml-virtgpu: REMOTE_CALL: fail if the backend returns an error * docs/backend/VirtGPU.md: indicate that the RAM+VRAM size is limed to 64 GB with libkrun * ggml-virtgpu: turn off clang-format header ordering for some of the files Compilation breaks when ordered alphabetically. * ggml-virtgpu: clang-format * ggml-virtgpu/backend/shared/api_remoting: better comments for the APIR return codes | 6 个月前 | |
docs : update ZenDNN docs for Q8 support (#23791) * docs zendnn added information about Q8 support * docs zendnn rm unnecessary data * docs update, links to ZenDNN docs provided * docs zenDNN update: clarified explanation * docs zenDNN update: one more explanation clarified --------- Co-authored-by: plotnikov.v10 <plotnikov.v10@wb.ru> | 3 个月前 | |
ggml-zendnn : add ZenDNN backend for AMD CPUs (#17690) * ggml-zennn: add ZenDNN backend support * ggml-zendnn : address ZenDNN backend review fixes and suggestions * docs : apply blockquote syntax to ZenDNN docs --------- Co-authored-by: Manoj Kumar <mkumar@zettabolt.com> | 9 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 1 个月前 | ||
| 1 年前 | ||
| 5 个月前 | ||
| 3 个月前 | ||
| 2 个月前 | ||
| 1 个月前 | ||
| 30 天前 | ||
| 1 个月前 | ||
| 6 个月前 | ||
| 3 个月前 | ||
| 9 个月前 |