一、StreamPark简介
StreamPark（原StreamX）是一个流处理应用程序开发管理框架。初衷是让流处理更简单，旨在轻松构建和管理流处理应用程序，提供使用 Apache Flink 和 Apache Spark 编写流处理应用程序的开发框架，未来将支持更多其他引擎。同时，StreamPark 提供了一个流处理应用管理平台，核心能力包括但不限于应用开发、调试、交互查询、部署、运维、实时数仓等。
官方网站：https://streampark.apache.org
二、快速安装使用
1. 使用docker-compose安装
1.1 安装docker
首先要安装docker，要求如下：
Docker 1.13.1+
Docker Compose 1.28.0+
1.2 安装docker-compose
docker-compose安装方式有很多，这里在尝试pip安装后，启动StreamPark后频繁报错，所以推荐二进制安装方式。
-
首先下载docker-compose文件，使用daocloud下载会快点，安装最新稳定版v2.16.0
sudo curl -L https://get.daocloud.io/docker/compose/releases/download/2.16.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose -
接着将docker-compose添加可执行权限
sudo chmod +x /usr/local/bin/docker-compose -
查询版本
docker-compose --version
###1.3 下载docker-compose.yaml和.env
官方给的指令：
wget https://github.com/apache/incubator-streampark/blob/dev/deploy/docker/docker-compose.yaml
wget https://github.com/apache/incubator-streampark/blob/dev/deploy/docker/.env
这里我下载到的文件里面是html代码，且下载麻烦，下面直接给文件内容，创建两个文件就行：
sudo mkdir -p streampark
sudo vim docker-compose.yaml
- docker-compose.yaml
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: '3.8'
services:
streampark-console:
image: apache/streampark:latest
command: ${RUN_COMMAND}
ports:
- 10000:10000
env_file: .env
volumes:
- flink:/streampark/flink/${FLINK}
- /var/run/docker.sock:/var/run/docker.sock
- /etc/hosts:/etc/hosts:ro
- ~/.kube:/root/.kube:ro
privileged: true
restart: unless-stopped
networks:
- streampark
flink-jobmanager:
image: ${FLINK_IMAGE}
ports:
- "8081:8081"
command: jobmanager
volumes:
- flink:/opt/flink
env_file: .env
restart: unless-stopped
privileged: true
networks:
- streampark
flink-taskmanager:
image: ${FLINK_IMAGE}
depends_on:
- flink-jobmanager
command: taskmanager
deploy:
replicas: 1
env_file: .env
restart: unless-stopped
privileged: true
networks:
- streampark
networks:
streampark:
driver: bridge
volumes:
flink:
这里要注意按照自己的需求调整配置，特别注意端口不能和现有服务端口冲突文章来源：https://www.toymoban.com/news/detail-440398.html
- .env
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
TZ=Asia/Shanghai
SPRING_PROFILES_ACTIVE=h2 #mysql, pgsql
# If use mysql or pgsql, please set the following parameters
#SPRING_DATASOURCE_URL=jdbc:mysql://localhost:3306/streampark?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=false&useJDBCCompliantTimezoneShift=true&useLegacyDatetimeCode=false&serverTimezone=GMT%2B8
#SPRING_DATASOURCE_URL=jdbc:postgresql://localhost:5432/streampark?stringtype=unspecified
#SPRING_DATASOURCE_USERNAME=root
#SPRING_DATASOURCE_PASSWORD=streampark
FLINK=flink1.14.5
FLINK_IMAGE=flink:1.14.5-scala_2.12
RUN_COMMAND='/bin/sh -c "wget -P lib https://repo1.maven.org/maven2/com/mysql/mysql-connector-j/8.0.31/mysql-connector-j-8.0.31.jar && bash bin/startup.sh"'
JOB_MANAGER_RPC_ADDRESS=flink-jobmanager
1.4 启动容器
docker-compose up -d
1.5 查看服务
- 访问StreamPark http://yourip:10000，这里的yourip是指宿主机的ip
- 默认账号和密码是admin 和 streampark
- 访问StreamPark http://yourip:8081，同上
- 这里的端口是和配置文件中的一致的，如果有改动，改成对应的端口即可
1.6 快速开始
请见官方文档：[StreamPark-快速开始](https://streampark.apache.org/zh-CN/docs/user-guide/quick-start)
2. 基于源码构建镜像进行StreamPark部署
未完待续。
三、常见报错
未完待续。。。
报错记录不在，后续有空更新文章来源地址https://www.toymoban.com/news/detail-440398.html
到了这里，关于流计算开发平台StreamPark安装的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
