Allow importing multi-file GGUF models

### What is the issue? Currently Ollama can [import GGUF files](https://github.com/ollama/ollama/blob/main/docs/import.md). However, larger models are sometimes split into separate files. Ollama should support loading multiple GGUF files similar to loading safetensor files. ### Workaround The `ollama create` command now supports importing multi-file GGUF models. For example: ```sh hf download unsloth/Qwen3-Coder-Next-GGUF \ --include "Q4_1/*.gguf" \ --local-dir ./qwen3-coder-next-gguf cd qwen3-coder-next-gguf/Q4_1 ollama create qwen3-coder-next:q4_1 ollama run qwen3-coder-next:q4_1 ``` Once you have created the model, you can remove the temporary download directory.

Posted by jmorganca on Jun 23, 2024, 9:45 PM

Tags

• support (Software)

Related issues

• Add 65536 size support for maps
• TypeScript 7.1 Iteration Plan
• nvim-treeconfig: upstream nvim-treesitter
• Add the "action" and "method" attributes to <button>
• [FEATURE] Support multiple Connector accounts (same connector, different accounts) in Claude and Claude Code on the web (claude.ai/code)
• > We were imposed GitHub for so many exercises!