模型支持列表

NLP

masked_language_modeling

text_generation

模型 model	模型规格 type	数据集 dataset	评估指标 metric	评估得分 score	配置 config
llama2	llama2_7b llama2_13b llama2_7b_lora llama2_13b_lora llama2_70b	alpaca	PPL / EM / F1	6.58 / 39.6 / 60.5 6.14 / 27.91 / 44.23 - - -	configs
glm2	glm2_6b glm2_6b_lora	ADGEN	BLEU-4 / Rouge-1 / Rouge-2 / Rouge-l	7.47 / 30.78 / 7.07 / 24.77 7.23 / 31.06 / 7.18 / 24.23	configs
glm3	glm3_6b	ADGEN	-	-	configs
gpt2	gpt2_small gpt2_13b	wikitext-2	-	-	configs
baichuan2	baichuan2_7b baichuan2_13b baichuan2_7b_lora baichuan2_13b_lora	belle	-	-	configs
codegeex2	codegeex2_6b	CodeAlpaca	-	-	configs
codellama	codellama_34b	CodeAlpaca	-	-	configs
deepseek	deepseek_33b	-	-	-	configs
internlm	internlm_7b internlm_20b	alpaca	-	-	configs
mixtral	mixtral-8x7b	wikitext-2	-	-	configs
qwen	qwen_7b qwen_14b qwen_7b_lora qwen_14b_lora	alpaca	C-Eval	63.3 72.13 - -	configs
qwen1.5	qwen1.5-14b qwen1.5-72b	-	-	-	configs
wizardcoder	wizardcoder_15b	CodeAlpaca	MBPP Pass@1	50.8	configs
yi	yi_6b yi_34b	alpaca_gpt4_data_zh	-	-	configs

LLM大模型能力支持一览

模型 \ 特性	低参微调	边训边评	Flash Attention	并行推理	流式推理	Chat	多轮对话
Llama2-7B/13B/70B	Lora	PPL	√	dp/mp	√	√	√
GLM2-6B	Lora/P-TuningV2	PPL/Bleu/Rouge	√	dp/mp	√	√	√
GLM3-6B	Lora	×	√	dp/mp	√	√	√
CodeGeex2-6B	×	PPL/Bleu/Rouge	√	dp/mp	√	√	√
CodeLlama-34B	Lora	pass@1	√	dp/mp	√	√	×
GPT2-128m/13B	Lora	PPL	√	dp/mp	√	×	×
BaiChuan2-7B/13B	Lora	PPL	√	dp/mp	√	√	√
Qwen-7B/14B	√	×	√	dp/mp	√	√	√
Qwen1.5-14B	×	×	×	dp/mp	√	√	√
Qwen1.5-72B	Lora	×	√	dp/mp	√	√	√
InternLM-7B/20B	Lora	PPL	√	dp/mp	√	√	√
Wizardcoder-15B	×	PPL	×	dp/mp	√	√	√
Deepseek-33B	×	×	×	×	×	√	×
Mixtral-8×7B	×	×	√	√	×	×	×
Yi-6B	Lora	×	√	dp/mp	√	√	√
Yi-34B	×	×	×	dp/mp	√	×	×