llama.cpp/example/embedding

This example demonstrates generate high-dimensional embedding vector of a given text with llama.cpp.

Quick Start

To get started right away, run the following command, making sure to use the correct path for the model you have:

Unix-based systems (Linux, macOS, etc.):

./llama-embedding -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>/dev/null

Windows:

llama-embedding.exe -m ./path/to/model --pooling mean --log-disable -p "Hello World!" 2>$null

The above command will output space-separated float values.

extra parameters

--embd-normalize $integer$

$integer$	description	formula
$-1$	none
$0$	max absolute int16	$\frac{32760\ast x_i}{\max |x_i|}$
$1$	taxicab	$\frac{x_i}{\sum |x_i|}$
$2$	euclidean (default)	$\frac{x_i}{\sqrt{\sum x_i^2}}$
$>2$	p-norm	$\frac{x_i}{\sqrt[p]{\sum |x_i{|}^p}}$

--embd-output-format ${}^{\prime }strin{g}^{\prime }$

${}^{\prime }strin{g}^{\prime }$	description
''	same as before	(default)
'array'	single embeddings	$[[x_1,...,x_n]]$
	multiple embeddings	$[[x_1,...,x_n],[x_1,...,x_n],...,[x_1,...,x_n]]$
'json'	openai style
'json+'	add cosine similarity matrix
'raw'	plain text output

--embd-separator $"string"$

$"string"$
"\n"	(default)
"<#embSep#>"	for example
"<#sep#>"	other example

examples

Unix-based systems (Linux, macOS, etc.):

./llama-embedding -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null

Windows:

llama-embedding.exe -p 'Castle<#sep#>Stronghold<#sep#>Dog<#sep#>Cat' --pooling mean --embd-separator '<#sep#>' --embd-normalize 2  --embd-output-format '' -m './path/to/model.gguf' --n-gpu-layers 99 --log-disable 2>/dev/null