macOS、Windows、Linux、Docker等各个平台通过ollama一键部署谷歌最新开源的gemma大模型，免费开源离线部署使用超越chatgpt4。谷歌最强开源大模型亮相！Gemini技术下放，笔记本就能跑，可商用。
谷歌12天连放三个大招
9日-宣布其最强大模型Gemini Ultra免费用，于2023年12月发布时在MMLU（大规模多任务语言理解）测评上超过人类专家，在32个多模态基准中取得30个SOTA（当前最优效果），几乎全方位超越GPT-4，向OpenAI发起强势一击。
16日-放出大模型“核弹”Gemini 1.5，并将上下文窗口长度扩展到100万个tokens。Gemini 1.5 Pro可一次处理1小时的视频、11小时的音频、超过3万行代码或超过70万字的代码库，向OpenAI还没发布的GPT-5发起挑战。
21日-突然放出开源模型Gemma，谷歌在被“抢头条”后，一举将采用与Gemini相同研究和技术的Gemma开源，一是狙击Llama 2等开源模型，二是为生成式AI的应用开发者带来福音。 谷歌这动作之密集、行动之迅速，似乎在向抢了自己风头的OpenAI宣战。 一夜之间，Gemma系列正式上线，全面对外开放。
它采用Gemini同款技术架构，主打开源和轻量级，免费可用、模型权重开源、允许商用，同时笔记本可跑。
共有2B和7B两个版本，性能全面超越开源标杆Llama 2。 每种规模都有预训练和指令微调版本，可在Kaggle、Colab Notebook、Google Cloud中访问。
而且支持JAX、PyTorch和TensorFlow通过原生Keras 3.0进行推理和监督式微调（SFT），适应多种开发需求和环境。得益于对JAX的支持，它还能进行快速推理。
目前模型也同步上线Hugging Chat，可在线体验试玩。
发布几个小时里，Gemma火速成为圈内最热话题，成为议论焦点。
多性能超越同规模开源模型 具体来看Gemma的技术报告。
在18个任务中，Gemma在11个任务上表现优于同规模优质开源模型。 Gemma包含两种规格。
7B版本参数量约78亿，面向GPU和TPU上的高效部署和开发，2B版本参数量约25亿，用于CPU和端侧应用程序。
它基于Transformer解码器架构，关键模型参数如下。
相比于基础Transformer，Gemma进行了一些升级。
7B版本使用多头注意力机制，2B版本使用多查询注意力机制。
在每一层中使用旋转位置嵌入代替绝对位置嵌入；使用GeGLU激活函数替代标准ReLU非线性。
同时对每一个子层的输入和输出都进行归一化。
Gemma 2B/7B分别使用了2T和6T token进行训练，主要来自网络文档、数学和代码，不过这些数据不是多模态的。
为了兼容，谷歌使用了Gemini的SentencePiece tokenizer子集，它可以分割数字，不删除额外的空格，并对未知token进行字节级编码。
大神卡帕西关注了Tokenizer部分，他表示，Gemma的tokenizer和Llama 2的不同，但和GPT一致。
François Chollet认为Gemma最大的特点是谷歌拥有SOTA级测试集过滤机制，这意味着基准数据能相当准确地反映了模型在实际环境中的表现。
谷歌也报告了Gemma在MMLU等基准上的表现。
最后，谷歌还强调了Gemma的安全隐私性能。
实验数据显示Gemma不会存储敏感数据，但可能会记住一些潜在隐私数据。不过报告表示这个数据可能因为工具原因有所误报。
gemma：https://ollama.com/library/gemma Gemma is a family of lightweight, state-of-the-art open models built by Google DeepMind.
Note: this model requires Ollama 0.1.26 or later. Download it here. https://ollama.com/download
Gemma is a new open model developed by Google and its DeepMind team. It’s inspired by Gemini models at Google.
Gemma is available in both 2b and 7b parameter sizes:
ollama run gemma:2b ollama run gemma:7b (default) The models undergo training on a diverse dataset of web documents to expose them to a wide range of linguistic styles, topics, and vocabularies. This includes code to learn syntax and patterns of programming languages, as well as mathematical text to grasp logical reasoning.
To ensure the safety of the model, the team employed various data cleaning and filtering techniques, including rigorous filtering for CSAM (child sexual abuse material), sensitive data filtering, and filtering based on content quality in compliance with Google’s policies.
Reference Google Gemma：https://ai.google.dev/gemma/docs/model_card
latest 5.2GB • 430ed3535049 • 12 hours ago ollama run gemma
2b 1.7GB • ff0200cc2d50 • 11 hours ago ollama run gemma:2b
text 5.2GB • af57093b878e • 9 hours ago ollama run gemma:text
instruct 5.2GB • 430ed3535049 • 12 hours ago ollama run gemma:instruct
7b 5.2GB • 430ed3535049 • 12 hours ago ollama run gemma:7b
2b-instruct 1.7GB • ff0200cc2d50 • 11 hours ago ollama run gemma:2b-instruct
2b-text 1.7GB • 8e7dbe189b8c • 11 hours ago ollama run gemma:2b-text
7b-instruct 5.2GB • 430ed3535049 • 12 hours ago ollama run gemma:7b-instruct
7b-text 5.2GB • af57093b878e • 9 hours ago ollama run gemma:7b-text
2b-instruct-q4_0 1.7GB • ff0200cc2d50 • 11 hours ago ollama run gemma:2b-instruct-q4_0
2b-instruct-q4_1 1.8GB • 0531ce560c52 • 11 hours ago ollama run gemma:2b-instruct-q4_1
2b-instruct-q5_0 1.9GB • bd0c49d488a7 • 11 hours ago ollama run gemma:2b-instruct-q5_0
2b-instruct-q5_1 2.1GB • 8494d3a93bb0 • 11 hours ago ollama run gemma:2b-instruct-q5_1
2b-instruct-q8_0 2.7GB • a1bea2062b57 • 11 hours ago ollama run gemma:2b-instruct-q8_0
2b-instruct-q2_K 1.3GB • 4bd3d747b45a • 11 hours ago ollama run gemma:2b-instruct-q2_K
2b-instruct-q3_K_S 1.4GB • e5ea800f0842 • 11 hours ago ollama run gemma:2b-instruct-q3_K_S
2b-instruct-q3_K_M 1.5GB • 16b226fa6db2 • 11 hours ago ollama run gemma:2b-instruct-q3_K_M
2b-instruct-q3_K_L 1.6GB • e79e55e83410 • 11 hours ago ollama run gemma:2b-instruct-q3_K_L
2b-instruct-q4_K_S 1.7GB • 3cbacca7aba1 • 11 hours ago ollama run gemma:2b-instruct-q4_K_S
2b-instruct-q4_K_M 1.8GB • 64a29cbcb55c • 11 hours ago ollama run gemma:2b-instruct-q4_K_M
2b-instruct-q5_K_S 1.9GB • 08c46e51eec0 • 11 hours ago ollama run gemma:2b-instruct-q5_K_S
2b-instruct-q5_K_M 2.0GB • 08e646708d53 • 11 hours ago ollama run gemma:2b-instruct-q5_K_M
2b-instruct-q6_K 2.2GB • be48f404caec • 11 hours ago ollama run gemma:2b-instruct-q6_K
2b-instruct-fp16 4.5GB • 4a53019167fe • 11 hours ago ollama run gemma:2b-instruct-fp16
2b-text-q4_0 1.7GB • 8e7dbe189b8c • 11 hours ago ollama run gemma:2b-text-q4_0
2b-text-q4_1 1.8GB • 0f5defc9d92c • 11 hours ago ollama run gemma:2b-text-q4_1
2b-text-q5_0 1.9GB • ea2612176b5f • 11 hours ago ollama run gemma:2b-text-q5_0
2b-text-q5_1 2.1GB • 8991086c344d • 11 hours ago ollama run gemma:2b-text-q5_1
2b-text-q8_0 2.7GB • 5bffeae39c52 • 11 hours ago ollama run gemma:2b-text-q8_0
2b-text-q2_K 1.3GB • 706d4ab3e4b9 • 11 hours ago ollama run gemma:2b-text-q2_K
2b-text-q3_K_S 1.4GB • e20ed30a3bd0 • 11 hours ago ollama run gemma:2b-text-q3_K_S
2b-text-q3_K_M 1.5GB • 2f3d7ff15ab1 • 11 hours ago ollama run gemma:2b-text-q3_K_M
2b-text-q3_K_L 1.6GB • a6719f63fd68 • 11 hours ago ollama run gemma:2b-text-q3_K_L
2b-text-q4_K_S 1.7GB • 979eec9c5947 • 11 hours ago ollama run gemma:2b-text-q4_K_S
2b-text-q4_K_M 1.8GB • 42453f05d9d7 • 10 hours ago ollama run gemma:2b-text-q4_K_M
2b-text-q5_K_S 1.9GB • 504bda6a88b7 • 10 hours ago ollama run gemma:2b-text-q5_K_S
2b-text-q5_K_M 2.0GB • 5e528905a274 • 10 hours ago ollama run gemma:2b-text-q5_K_M
2b-text-q6_K 2.2GB • 3c8c53aac289 • 10 hours ago ollama run gemma:2b-text-q6_K
2b-text-fp16 4.5GB • 6ce2a676d014 • 11 hours ago ollama run gemma:2b-text-fp16
7b-instruct-q4_0 5.2GB • 430ed3535049 • 12 hours ago ollama run gemma:7b-instruct-q4_0
7b-instruct-q4_1 5.7GB • e1e4a2e0c8ee • 12 hours ago ollama run gemma:7b-instruct-q4_1
7b-instruct-q5_0 6.2GB • de923fde2f26 • 12 hours ago ollama run gemma:7b-instruct-q5_0
7b-instruct-q5_1 6.7GB • 2e51588baf45 • 12 hours ago ollama run gemma:7b-instruct-q5_1
7b-instruct-q8_0 9.1GB • 61d0f0df3637 • 12 hours ago ollama run gemma:7b-instruct-q8_0
7b-instruct-q2_K 3.7GB • 831e95226882 • 11 hours ago ollama run gemma:7b-instruct-q2_K
7b-instruct-q3_K_S 4.2GB • e4aea70c287e • 11 hours ago ollama run gemma:7b-instruct-q3_K_S
7b-instruct-q3_K_M 4.6GB • 92dd270cb673 • 11 hours ago ollama run gemma:7b-instruct-q3_K_M
7b-instruct-q3_K_L 4.9GB • 1fa1f8e1a003 • 11 hours ago ollama run gemma:7b-instruct-q3_K_L
7b-instruct-q4_K_S 5.2GB • bb7b8325814d • 11 hours ago ollama run gemma:7b-instruct-q4_K_S
7b-instruct-q4_K_M 5.5GB • d9c26a968eb4 • 11 hours ago ollama run gemma:7b-instruct-q4_K_M
7b-instruct-q5_K_S 6.2GB • 42630cbe71a2 • 11 hours ago ollama run gemma:7b-instruct-q5_K_S
7b-instruct-q5_K_M 6.3GB • 556abeb39bfd • 11 hours ago ollama run gemma:7b-instruct-q5_K_M
7b-instruct-q6_K 7.2GB • 5c7aded0b8bd • 11 hours ago ollama run gemma:7b-instruct-q6_K
7b-instruct-fp16 17GB • f689ad351c8d • 11 hours ago ollama run gemma:7b-instruct-fp16
7b-text-q4_0 5.2GB • af57093b878e • 9 hours ago ollama run gemma:7b-text-q4_0
7b-text-q4_1 5.7GB • a7adb7322d8b • 9 hours ago ollama run gemma:7b-text-q4_1
7b-text-q5_0 6.2GB • fd6dc849a9f8 • 9 hours ago ollama run gemma:7b-text-q5_0
7b-text-q5_1 6.7GB • 938308342124 • 9 hours ago ollama run gemma:7b-text-q5_1
7b-text-q8_0 9.1GB • 1b7b1e5e2f98 • 9 hours ago ollama run gemma:7b-text-q8_0
7b-text-q2_K 3.7GB • e43e18a484ed • 9 hours ago ollama run gemma:7b-text-q2_K
7b-text-q3_K_S 4.2GB • f40e21d459ff • 9 hours ago ollama run gemma:7b-text-q3_K_S
7b-text-q3_K_M 4.6GB • ffd8ea66cf1f • 9 hours ago ollama run gemma:7b-text-q3_K_M
7b-text-q3_K_L 4.9GB • 1676d1f86165 • 9 hours ago ollama run gemma:7b-text-q3_K_L
7b-text-q4_K_S 5.2GB • 51854f1241b3 • 8 hours ago ollama run gemma:7b-text-q4_K_S
7b-text-q4_K_M 5.5GB • 5ec372e05241 • 8 hours ago ollama run gemma:7b-text-q4_K_M
7b-text-q5_K_S 6.2GB • df449b30e2e1 • 8 hours ago ollama run gemma:7b-text-q5_K_S
7b-text-q5_K_M 6.3GB • dc8f812e49c7 • 8 hours ago ollama run gemma:7b-text-q5_K_M
7b-text-q6_K 7.2GB • 9e7cfd3fab5b • 8 hours ago ollama run gemma:7b-text-q6_K
7b-text-fp16 16GB • 1f1e9df10872 • 9 hours ago ollama run gemma:7b-text-fp16
https://ollama.com/library/gemma/tags
Ollama
https://discord.gg/ollama
Get up and running with large language models locally.
macOS
Download
Windows preview
Download
Linux
curl -fsSL https://ollama.com/install.sh | sh
Manual install instructions
Docker
The official Ollama Docker image ollama/ollama is available on Docker Hub.
Libraries
ollama-pythonollama-js
Quickstart
To run and chat with Llama 2:
ollama run llama2
Model library
Ollama supports a list of models available on ollama.com/library
Here are some example models that can be downloaded:
ModelParametersSizeDownloadLlama 27B3.8GBollama run llama2Mistral7B4.1GBollama run mistralDolphin Phi2.7B1.6GBollama run dolphin-phiPhi-22.7B1.7GBollama run phiNeural Chat7B4.1GBollama run neural-chatStarling7B4.1GBollama run starling-lmCode Llama7B3.8GBollama run codellamaLlama 2 Uncensored7B3.8GBollama run llama2-uncensoredLlama 2 13B13B7.3GBollama run llama2:13bLlama 2 70B70B39GBollama run llama2:70bOrca Mini3B1.9GBollama run orca-miniVicuna7B3.8GBollama run vicunaLLaVA7B4.5GBollama run llavaGemma2B1.4GBollama run gemma:2bGemma7B4.8GBollama run gemma:7b
Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Customize a model
Import from GGUF
Ollama supports importing GGUF models in the Modelfile:
Create a file named Modelfile, with a FROM instruction with the local filepath to the model you want to import. FROM ./vicuna-33b.Q4_0.gguf
Create the model in Ollama ollama create example -f Modelfile
Run the model ollama run example
Import from PyTorch or Safetensors
See the guide on importing models for more information.
Customize a prompt
Models from the Ollama library can be customized with a prompt. For example, to customize the llama2 model:
ollama pull llama2
Create a Modelfile:
FROM llama2
# set the temperature to 1 [higher is more creative, lower is more coherent]
PARAMETER temperature 1
# set the system message
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
Next, create and run the model:
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
For more examples, see the examples directory. For more information on working with a Modelfile, see the Modelfile documentation.
CLI Reference
Create a model
ollama create is used to create a model from a Modelfile.
ollama create mymodel -f ./Modelfile
Pull a model
ollama pull llama2
This command can also be used to update a local model. Only the diff will be pulled.
Remove a model
ollama rm llama2
Copy a model
ollama cp llama2 my-llama2
Multiline input
For multiline input, you can wrap text with """:
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
Multimodal models
>>> What's in this image? /Users/jmorgan/Desktop/smile.png
The image features a yellow smiley face, which is likely the central focus of the picture.
Pass in prompt as arguments
$ ollama run llama2 "Summarize this file: $(cat README.md)"
Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
List models on your computer
ollama list
Start Ollama
ollama serve is used when you want to start ollama without running the desktop application.
Building
Install cmake and go:
brew install cmake go
Then generate dependencies:
go generate ./...
Then build the binary:
go build .
More detailed instructions can be found in the developer guide
Running local builds
Next, start the server:
./ollama serve
Finally, in a separate shell, run a model:
./ollama run llama2
REST API
Ollama has a REST API for running and managing models.
Generate a response
curl http://localhost:11434/api/generate -d '{
"model": "llama2",
"prompt":"Why is the sky blue?"
}'
Chat with a model
curl http://localhost:11434/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
See the API documentation for all endpoints.
Community Integrations
Web & Desktop
Bionic GPTEnchanted (macOS native)HTML UIChatbot UITypescript UIMinimalistic React UI for Ollama ModelsOpen WebUIOllamacbig-AGICheshire Cat assistant frameworkAmicachatdOllama-SwiftUIMindMacNextJS Web Interface for OllamaMsty
Terminal
otermEllama Emacs clientEmacs clientgen.nvimollama.nvimollama-chat.nvimogpt.nvimgptel Emacs clientOatmealcmdhtenerellm-ollama for Datasette’s LLM CLI.ShellOracle
Database
MindsDB
Package managers
PacmanHelm Chart
Libraries
LangChain and LangChain.js with exampleLangChainGo with exampleLangChain4j with exampleLlamaIndexLangChain4jLiteLLMOllamaSharp for .NETOllama for RubyOllama-rs for RustOllama4j for JavaModelFusion Typescript LibraryOllamaKit for SwiftOllama for DartOllama for LaravelLangChainDartSemantic Kernel - PythonHaystackElixir LangChainOllama for R - rollamaOllama-ex for ElixirOllama Connector for SAP ABAP
Mobile
EnchantedMaid
Extensions & Plugins
Raycast extensionDiscollama (Discord bot inside the Ollama discord channel)ContinueObsidian Ollama pluginLogseq Ollama pluginDagger ChatbotDiscord AI BotOllama Telegram BotHass Ollama ConversationRivet pluginLlama Coder (Copilot alternative using Ollama)Obsidian BMO Chatbot pluginCopilot for Obsidian pluginObsidian Local GPT pluginOpen Interpretertwinny (Copilot and Copilot chat alternative using Ollama)Wingman-AI (Copilot code and chat alternative using Ollama and HuggingFace)Page Assist (Chrome Extension)
参考资料： https://github.com/ollama/ollama
参考链接
发表评论