安装hugginface-hub

$ pip install huggingface-hub

下载HuggingFace上的模型

本次使用此模型Moistral-11B-v4-Q4_K_M.gguf。执行如下命令

$ huggingface-cli download TheDrummer/Moistral-11B-v4-GGUF Moistral-11B-v4-Q4_K_M.gguf --local-dir downloads  --local-dir-use-symlinks False

创建Modelfile.txt模型文件

Modelfile语法请参考modelfile.md。

Modelfile.txt的内容如下：

FROM ./downloads/Moistral-11B-v4-Q4_K_M.gguf

创建模型

$ ollama create Moistral-11B-v4:Q4 -f Modelfile.txt
transferring model data 
using existing layer sha256:6d05a77c8266838b95b6e001d358e7ec32b6ca7e0d6d235cc895f5fab8234efa 
creating new layer sha256:1a282d2de7be1dadca24301f4ccccbcc851201307e171c532788129fba125d31 
writing manifest 
success

查看模型是否存在

$ ollama list
NAME                                	ID          	SIZE  	MODIFIED       
Moistral-11B-v4:Q4                  	34b8f0b14cc2	6.5 GB	31 seconds ago	
sunapi386/llama-3-lexi-uncensored:8b	ec6c7923b45f	4.9 GB	5 weeks ago   	
llama2-uncensored:7b                	44040b922233	3.8 GB	5 weeks ago   	
dolphin-llama3:8b-256k              	9f4257eb39a8	4.7 GB	6 weeks ago