部署参考视频
通义千问-7B-预训练-模型库
一、部署1
1.1 打开通义千问-7B-预训练-模型库-选择资源
1.2 使用Netbook
弹出新页面
2.1 运行
2.2 复制脚本
from modelscope import AutoModelForCausalLM, AutoTokenizer
from modelscope import GenerationConfig
import datetime
print("启动时间：" + str(datetime.datetime.now()))
tokenizer = AutoTokenizer.from_pretrained("qwen/Qwen-7B-Chat", revision = 'v1.0.5',trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("qwen/Qwen-7B-Chat", revision = 'v1.0.5',device_map="auto",offload_folder="offload_folder", trust_remote_code=True,fp16 = True).eval()
model.generation_config = GenerationConfig.from_pretrained("Qwen/Qwen-7B-Chat",revision = 'v1.0.5', trust_remote_code=True) # 可指定不同的生成长度、top_p等相关超参
model.float()
print("开始执行：" + str(datetime.datetime.now()))
response, history = model.chat(tokenizer, "你好", history=None)
print(response)
print("第一个问题处理完毕：" + str(datetime.datetime.now()))
response, history = model.chat(tokenizer, "浙江的省会在哪里？", history=history)
print(response)
print("第二个问题处理完毕：" + str(datetime.datetime.now()))
response, history = model.chat(tokenizer, "它有什么好玩的景点", history=history)
print(response)
print("第三个问题处理完毕：" + str(datetime.datetime.now()))
2.2.1 问题1 ：ImportError: This modeling file requires the following packages that were not found in your environment: transformers_stream_generator. Run pip install transformers_stream_generator
解决方法pip install transformers_stream_generator
这就好了，重新运行下
ValueError: The current device_map had weights offloaded to the disk. Please provide an offload_folder for them. Alternatively, make sure you have safetensors installed if the model you are using offers the weights in this format.
参照这哥们的
https://zhuanlan.zhihu.com/p/649272911
2.3 查看结果
3.其他作者的demo
3.1 克隆代码
git clone https://gitee.com/JokerBao/Qwen-7B-FastWeb.git
3.2 加载依赖
pip install -r requirements.txt
3.3 运行代码
python Qwen_demo.py
二、部署2
1.1 启动服务器
1.2 使用官方提供的demo
拉代码
git clone https://github.com/QwenLM/Qwen-7B.git
1.2 为了拉去大模型，更新LFS
apt-get update
apt-get install git-lfs
2.1 拉千问模型文件
git clone https://www.modelscope.cn/qwen/Qwen-7B-Chat.git
初始化一下
git init
git lfs install
2.1.1模型下载成功
2.1.2 安装依赖
为了方便我把模型移动到一开始的文件夹里面
pip install -r requirements.txt
也可以使用web依赖
pip install -r requirements_web_demo.txt
3.1 修改web.demo.py地址
然后ctrl+s 保存
4 运行
python web_demo.py
5 成功
5开启量化
pip install bitsandbytes
添加依赖
from transformers import BitsAndBytesConfig
import torch
添加文章来源：https://www.toymoban.com/news/detail-688251.html
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type='nf4',
bnb_4bit_compute_dtype=torch.bfloat16)
文章来源地址https://www.toymoban.com/news/detail-688251.html
到了这里，关于通义千问部署搭建的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
