背景
概念
Flowise
Flowise 是一个开源的用户界面可视化工具,它允许用户通过拖放的方式来构建自定义的大型语言模型(LLM)流程。
Flowise基于LangChain.js,是一个非常先进的图形用户界面,用于开发基于LLM的应用程序。Flowise还支持Docker和NodeJS,可以在本地或者在容器中运行。如果有需求修改相关代码,可以本地部署运行。
Langflow
Langflow 是一个用于创建和运行语言模型流程的网络应用程序。
Langflow是一个动态图,每个节点都是一个可执行单元¹。它的模块化和交互式设计促进了快速的实验和原型制作。Langflow还提供了一种无代码的AI生态系统,可以无缝地与您的团队熟悉和喜欢的工具和堆栈集成。
Localai
LocalAI是一个免费的、开源的OpenAI替代品。它是一个与OpenAI API规范兼容的REST API,允许你在本地或者在本地网络上使用消费级硬件运行大型语言模型(LLM)。它支持多个与ggml格式兼容的模型系列。
1.Flowise安装部署
git
https://github.com/FlowiseAI/Flowise
官网
FlowiseAI - Build LLMs Apps Easily
docker部署
[root@master docker]# git clone https://github.com/FlowiseAI/Flowise.git
[root@master langchain_wk]# ls
Flowise
[root@master langchain_wk]# cd Flowise/
[root@master Flowise]# ls
artillery-load-test.yml babel.config.js CODE_OF_CONDUCT-ZH.md CONTRIBUTING-ZH.md Dockerfile LICENSE.md packages README-ZH.md
assets CODE_OF_CONDUCT.md CONTRIBUTING.md docker images package.json README.md turbo.json
[root@master Flowise]# clear
[root@master Flowise]# ls
artillery-load-test.yml babel.config.js CODE_OF_CONDUCT-ZH.md CONTRIBUTING-ZH.md Dockerfile LICENSE.md packages README-ZH.md
assets CODE_OF_CONDUCT.md CONTRIBUTING.md docker images package.json README.md turbo.json
[root@master Flowise]# cd docker/
[root@master docker]# ls
docker-compose.yml Dockerfile README.md
[root@master docker]# ls -alh
总用量 24K
drwxr-xr-x 2 root root 4.0K 11月 21 13:21 .
drwxr-xr-x 9 root root 4.0K 11月 21 13:21 ..
-rw-r--r-- 1 root root 988 11月 21 13:21 docker-compose.yml
-rw-r--r-- 1 root root 441 11月 21 13:21 Dockerfile
-rw-r--r-- 1 root root 675 11月 21 13:21 .env.example
-rw-r--r-- 1 root root 1.3K 11月 21 13:21 README.md
[root@master docker]# cp .env.example .env
[root@master docker]# vim .env
[root@master docker]#
修改.env配置文件
PORT=3090
DATABASE_PATH=/root/.flowise
APIKEY_PATH=/root/.flowise
SECRETKEY_PATH=/root/.flowise
LOG_PATH=/root/.flowise/logs
# NUMBER_OF_PROXIES= 1
# DATABASE_TYPE=postgres
# DATABASE_PORT=""
# DATABASE_HOST=""
# DATABASE_NAME="flowise"
# DATABASE_USER=""
# DATABASE_PASSWORD=""
# FLOWISE_USERNAME=user
# FLOWISE_PASSWORD=1234
# FLOWISE_SECRETKEY_OVERWRITE=myencryptionkey
DEBUG=true
# LOG_LEVEL=debug (error | warn | info | verbose | debug)
# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash
# LANGCHAIN_TRACING_V2=true
# LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
# LANGCHAIN_API_KEY=your_api_key
# LANGCHAIN_PROJECT=your_project
docker-compose启动
[root@master docker]# docker-compose up -d
WARNING: The FLOWISE_USERNAME variable is not set. Defaulting to a blank string.
WARNING: The FLOWISE_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_TYPE variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_PORT variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_HOST variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_NAME variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_USER variable is not set. Defaulting to a blank string.
WARNING: The DATABASE_PASSWORD variable is not set. Defaulting to a blank string.
WARNING: The FLOWISE_SECRETKEY_OVERWRITE variable is not set. Defaulting to a blank string.
WARNING: The LOG_LEVEL variable is not set. Defaulting to a blank string.
WARNING: Found orphan containers (supabase-rest, supabase-studio, supabase-edge-functions, supabase-auth, realtime-dev.supabase-realtime, supabase-imgproxy, supabase-storage, supabase-meta, supabase-db, supabase-kong) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Pulling flowise (flowiseai/flowise:)...
latest: Pulling from flowiseai/flowise
96526aa774ef: Pull complete
3130715204cf: Pull complete
b06de8ab1c4f: Pull complete
90ef3ffc5156: Pull complete
018fd88f5f14: Pull complete
e405bd0a6e44: Pull complete
6678d95b4ce0: Pull complete
f0b6d32b7e1f: Pull complete
754763d6e863: Pull complete
db86a2a89c8a: Pull complete
Digest: sha256:348211f573335adca3fdcfa1d62345eadee65e19cc3d83301f96b0359f31dc03
Status: Downloaded newer image for flowiseai/flowise:latest
Creating docker_flowise_1 ... done
[root@master docker]# docker-compose logs -f
Attaching to docker_flowise_1
flowise_1 | 2023-11-21 05:30:05 [INFO]: Starting Flowise...
flowise_1 | 2023-11-21 05:30:06 [INFO]: ⚡️ [server]: Flowise Server is listening at 3090
flowise_1 | 2023-11-21 05:30:06 [INFO]: 📦 [server]: Data Source has been initialized!
这里不多做flowise的教程,可以参考相关官方文档。
2.Localai 部署
注意:这里默认用的是cpu,localai主推用CPU,LocalAI具有Rust后端,内存效率高,体积小,并且后端模型部署gRpc,至于gpu部署,需要不同的nvidia cuda版本,可根据自己的cuda版本拉取相关镜像,然后重新build镜像;
build镜像会非常,最好使用代理或更换国内相关系统镜像源,rust源,go源,pip源;
# docker images | grep nvcr.io/nvidia/cuda
nvcr.io/nvidia/cuda 11.2.1-cudnn8-devel-ubuntu20.04 b74328ac303f 5 months ago 7.72GB
nvcr.io/nvidia/cuda 11.4.2-cudnn8-devel-ubi8 260b885046fb 5 months ago 9.76GB
nvcr.io/nvidia/cuda 11.4.0-cudnn8-devel-centos7 32f9c23be6b7 5 months ago 9.69GB
nvcr.io/nvidia/cuda 11.4.2-cudnn8-runtime-ubi8 16112a9b5038 5 months ago 3.89GB
这里用的11.2.1-cudnn8-devel-ubuntu20.04;
GPU的部署和编译相对比较麻烦,可参考官方文档。
GPU编译方法
make BUILD_TYPE=cublas build
启动方法
# ./local-ai --debug=true --f16=true --autoload-galleries=true
3:17PM INF Starting LocalAI using 4 threads, with models path: /build/models
3:17PM INF LocalAI version: v1.40.0-22-g991ecce (991ecce00462db955934ec1293954b7d61c58ad2)
3:17PM DBG Model: bert-embeddings (config: {PredictionOptions:{Model:bert-MiniLM-L6-v2q4_0.bin Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:bert-embeddings F16:false Threads:0 Debug:false Roles:map[] Embeddings:true Backend:bert-embeddings TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})
3:17PM DBG Model: thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin (config: {PredictionOptions:{Model:wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_K_M.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin F16:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})
3:17PM DBG Extracting backend assets files to /tmp/localai/backend_data
┌───────────────────────────────────────────────────┐
│ Fiber v2.50.0 │
│ http://127.0.0.1:8080 │
│ (bound on host 0.0.0.0 and port 8080) │
│ │
│ Handlers ............ 74 Processes ........... 1 │
│ Prefork ....... Disabled PID ............. 10297 │
└───────────────────────────────────────────────────┘
MODEL GALLERY 包含了huggingface的项目,如果下载模型,可能回被墙,至于如何解决,请自行解决。
下载模型
[192.168.12.82]:62589 200 - POST /models/apply
3:41PM DBG Checking "bert-MiniLM-L6-v2q4_0.bin" exists and matches SHA
3:41PM DBG File "bert-MiniLM-L6-v2q4_0.bin" already exists and matches the SHA. Skipping download
3:41PM DBG Written config file /build/models/bert-embeddings.yaml
编码测试
3.Flowise文档知识库召回应用
文章来源:https://www.toymoban.com/news/detail-803005.html
转载请注明出处!文章来源地址https://www.toymoban.com/news/detail-803005.html
到了这里,关于Flowise+LocalAI部署--Agent应用的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!