docker搭建Grafana+Loki+Promtail日志分析

这篇具有很好参考价值的文章主要介绍了docker搭建Grafana+Loki+Promtail日志分析。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

环境搭建

1. Docker 安装

Docker 官网下载:Docker: Accelerated, Containerized Application Development

Note: 本人个人原因,在WIndows OS 安装Docker

docker搭建Grafana+Loki+Promtail日志分析

 2. 安装完成后,验证Docker Service

# CMD 管理员界面

# sc query com.docker.service   # 查询
# sc start com.docker.service   # 启动
# sc stop com.docker.service    # 停止

docker搭建Grafana+Loki+Promtail日志分析

 docker搭建Grafana+Loki+Promtail日志分析

# CMD 管理员界面

D:\>docker version    # 查询docker Version
Client:
 Cloud integration: v1.0.29
 Version:           20.10.21
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        baeda1f
 Built:             Tue Oct 25 18:08:16 2022
 OS/Arch:           windows/amd64
 Context:           default
 Experimental:      true

Server: Docker Desktop 4.15.0 (93002)
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:00:19 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

D:\>
验证 Docker 运行

# CMD 管理员界面

D:\>docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
2db29710123e: Pull complete
Digest: sha256:94ebc7edf3401f299cd3376a1669bc0a49aef92d6d2669005f9bc5ef028dc333
d8d1a816d728: Downloading [=====================================>             ]  67.05MB/89.7MB

Hello from Docker!
This message shows that your installation appears to be working correctly.
d8d1a816d728: Downloading [=====================================>             ]  66.52MB/89.7MB
To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/


D:\>docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

D:\>
D:\>docker ps -a
CONTAINER ID   IMAGE         COMMAND    CREATED         STATUS                     PORTS     NAMES
16218564a475   hello-world   "/hello"   2 minutes ago   Exited (0) 2 minutes ago             reverent_jemison

D:\>

3. 为了方便,推荐安装Git Bash

请自行安装Git Bash

4. 开始安装grafana

1. 获取grafana  image 方式:

方式一: cmd管理员窗口 

                D:\Soft\Docker>docker pull grafana/grafana

docker搭建Grafana+Loki+Promtail日志分析

方式二:Git Bash 拉取

              #docker pull grafana/grafana

docker搭建Grafana+Loki+Promtail日志分析

2. 由于直接从国外拉去镜像文件,一直卡在 pulling fs layer. 

解决方法: 更换镜像源 (Windows Docker 配置国内镜像源的两种方法

通过Docker-Desktop界面操作和修改daemon.json两种方法配置国内镜像源

# 添加以下源地址:	

"registry-mirrors": [
        "https://docker.mirrors.ustc.edu.cn",
        "https://registry.docker-cn.com",
        "http://hub-mirror.c.163.com",
        "https://mirror.ccs.tencentyun.com"
    ]

docker搭建Grafana+Loki+Promtail日志分析

至此 再次查看 Git Bash ,Grafana image 已下载完成。

docker搭建Grafana+Loki+Promtail日志分析

 3. 检查grafana image

# docker images 

docker搭建Grafana+Loki+Promtail日志分析

4. 创建docker container

Note:
# docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
-d: 后台运行容器,并返回容器ID;
-p: 指定端口映射,格式为:主机(宿主)端口:容器端口
--name="nginx-lb": 为容器指定一个名称;

# 创建Grafana container

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker run -d -p 3000:3000 --name=grafana grafana/grafana
9066857121a2ef1780b84c2f2135490f1bf85487ba7aa4ba7c80ec0e7de52008


# 列出所有在运行的容器信息
# docker ps

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker ps
CONTAINER ID   IMAGE             COMMAND     CREATED          STATUS          PORTS                    NAMES
9066857121a2   grafana/grafana   "/run.sh"   24 seconds ago   Up 22 seconds   0.0.0.0:3000->3000/tcp   grafana

至此,docker 内的 grafana 就部署完成了

5. 安装Vector

1. docker 搜索可安装的 image file

# docker search <image file name>

docker搭建Grafana+Loki+Promtail日志分析

2. 安装 Vector

#  docker pull timberio/vector:0.10.0-alpine 

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker pull timberio/vector:0.10.0-alpine

0.10.0-alpine: Pulling from timberio/vector
0.10.0-alpine: Pulling from timberio/vector
21c83c524219: Pulling fs layer
88943f7483ee: Pulling fs layer
24da72aaf03b: Pulling fs layer
4145aaf02e3f: Pulling fs layer
b16d307dc057: Pulling fs layer
4145aaf02e3f: Waiting
88943f7483ee: Verifying Checksum
88943f7483ee: Download complete
21c83c524219: Verifying Checksum
21c83c524219: Download complete
21c83c524219: Pull complete
4145aaf02e3f: Verifying Checksum
4145aaf02e3f: Download complete
88943f7483ee: Pull complete
24da72aaf03b: Verifying Checksum
24da72aaf03b: Download complete
b16d307dc057: Verifying Checksum
24da72aaf03b: Pull complete
4145aaf02e3f: Pull complete
b16d307dc057: Pull complete
Digest: sha256:593180c5498ce28ebe922b87780a4a18e240592e2f5621b457e9041af5656da2
Status: Downloaded newer image for timberio/vector:0.10.0-alpine
docker.io/timberio/vector:0.10.0-alpine

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker

查询已下载的 docker image

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker images
REPOSITORY        TAG             IMAGE ID       CREATED         SIZE
grafana/grafana   latest          83f377cc32a0   3 weeks ago     317MB
hello-world       latest          feb5d9fea6a5   15 months ago   13.3kB
timberio/vector   0.10.0-alpine   2ebd263245dd   2 years ago     80.7MB

安装 image: timberio/vector:0.10.0-alpine 容器 名为 vector 的容器

# Git Bash

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker create --name vector timberio/vector:0.10.0-alpine
abbd527ebd3dfe0b02a0c11d892bf12c7fda25478a354c2e9f5604004989d803

Docker-Compose 搭建Grafana+Loki+Promtail 环境 

前言

Loki 组成:

  1. loki : 主服务器,负责存储日志和处理查询

  2. promtail : 代理,负责收集日志并将其发送给 loki

  3. Grafana : Go语言开发的开源数据可视化工具,可以做数据监控和数据统计,带有告警功能

Loki : https://github.com/grafana/loki

搭建环境:

1. 安装Docker

# 查询docker 

magx@DESKTOP-C1MJB99 MINGW64 ~
$ type -a docker
docker is /c/Program Files/Docker/Docker/resources/bin/docker

2. docker-compose

1. 安装Grafana Loki

        # 环境准备
        git clone https://gitee.com/zhengqingya/docker-compose.git

        or 手动网页访问 https://gitee.com/zhengqingya/docker-compose.git 直接克隆下载即可。

docker搭建Grafana+Loki+Promtail日志分析

# Windows系统走此路径
cd docker-compose/Windows

# Liunx系统走此路径
cd docker-compose/Liunx

Note:
# 【 注:Liunx系统先授权,否则会报错:`cannot create directory '/var/lib/grafana/plugins': Permission denied` 】
chmod 777 $PWD/grafana_promtail_loki/grafana/data
chmod 777 $PWD/grafana_promtail_loki/grafana/log

2.  运行服务
# docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up -d

# 停止服务

Note:

1. 此命令一次性启动前检查Grafana, Loki. Promtail  3个image,如无image 下载安装后并启动。

2. 以下命令可看出 

Attaching to

grafana_promtail_loki_demo_grafana,

grafana_promtail_loki_demo_loki,

grafana_promtail_loki_demo_promtail

# CMD 管理员权限
# 下载的docker-compose-master.zip 解压路径为:d:\Soft\Soft-dev\Dokcer\

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up -d
[+] Running 4/4
 - Network grafana_promtail_loki_loki             Created                                                                                                                                                    0.7s
 - Container grafana_promtail_loki_demo_promtail  Started                                                                                                                                                    1.6s
 - Container grafana_promtail_loki_demo_grafana   Started                                                                                                                                                    3.2s
 - Container grafana_promtail_loki_demo_loki      Started                                                                                                                                                    2.7s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                    STATUS              CONFIG FILES
grafana_promtail_loki   running(3)          D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki\docker-compose-grafana-promtail-loki.yml

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                    NAMES
6f692c924ce1   grafana/grafana:latest          "/run.sh"                22 seconds ago   Up 18 seconds   0.0.0.0:3000->3000/tcp   grafana_promtail_loki_demo_grafana
369d843c4cbb   grafana/loki:latest             "/usr/bin/loki -conf…"   22 seconds ago   Up 18 seconds   0.0.0.0:3100->3100/tcp   grafana_promtail_loki_demo_loki
2393ebad644f   grafana/promtail:latest         "/usr/bin/promtail -…"   22 seconds ago   Up 19 seconds                            grafana_promtail_loki_demo_promtail
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   16 hours ago     Up 16 hours     0.0.0.0:80->80/tcp       vector03

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki down
[+] Running 4/4
 - Container grafana_promtail_loki_demo_promtail  Removed                                                                                                                                                    0.8s
 - Container grafana_promtail_loki_demo_grafana   Removed                                                                                                                                                    0.6s
 - Container grafana_promtail_loki_demo_loki      Removed                                                                                                                                                    6.0s
 - Network grafana_promtail_loki_loki             Removed                                                                                                                                                    0.7s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                STATUS              CONFIG FILES

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS        PORTS                NAMES
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   16 hours ago   Up 16 hours   0.0.0.0:80->80/tcp   vector03

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
# CMD 管理员权限
# 下载的docker-compose-master.zip 解压路径为:d:\Soft\Soft-dev\Dokcer\

d:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up
[+] Running 3/0
 - Container grafana_promtail_loki_demo_loki      Created                                                                                               0.0s
 - Container grafana_promtail_loki_demo_promtail  Created                                                                                               0.0s
 - Container grafana_promtail_loki_demo_grafana   Created                                                                                               0.0s
Attaching to grafana_promtail_loki_demo_grafana, grafana_promtail_loki_demo_loki, grafana_promtail_loki_demo_promtail
grafana_promtail_loki_demo_promtail  | Unable to parse config: /etc/promtail/promtail-docker-config.yaml: yaml: unmarshal errors:
grafana_promtail_loki_demo_promtail  |   line 14: field entry_parser not found in type scrapeconfig.plain
grafana_promtail_loki_demo_loki      | creating WAL folder at "/wal": mkdir wal: permission denied
grafana_promtail_loki_demo_loki      | error initialising module: ingester
grafana_promtail_loki_demo_loki      | github.com/grafana/dskit/modules.(*Manager).initModule
grafana_promtail_loki_demo_loki      |  /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:122
grafana_promtail_loki_demo_loki      | github.com/grafana/dskit/modules.(*Manager).InitModuleServices
grafana_promtail_loki_demo_loki      |  /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:92
grafana_promtail_loki_demo_loki      | github.com/grafana/loki/pkg/loki.(*Loki).Run
grafana_promtail_loki_demo_loki      |  /src/loki/pkg/loki/loki.go:363
grafana_promtail_loki_demo_loki      | main.main
grafana_promtail_loki_demo_loki      |  /src/loki/cmd/loki/main.go:105
grafana_promtail_loki_demo_loki      | runtime.main
grafana_promtail_loki_demo_loki      |  /usr/local/go/src/runtime/proc.go:250
grafana_promtail_loki_demo_loki      | runtime.goexit
grafana_promtail_loki_demo_loki      |  /usr/local/go/src/runtime/asm_amd64.s:1594
grafana_promtail_loki_demo_promtail exited with code 1
grafana_promtail_loki_demo_grafana   | logger=settings t=2023-01-05T15:52:55.818434359Z level=info msg="Starting Grafana" version=9.3.2 commit=21c1d14e91 branch=HEAD compiled=2022-12-14T10:40:18Z
grafana_promtail_loki_demo_grafana   | logger=settings t=2023-01-05T15:52:55.818883348Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini
grafana_promtail_loki_demo_grafana   | logger=settings t=2023-01-05T15:52:55.81891629Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini

通过Docker-compose  一起启动

grafana_promtail_loki_demo_grafana,

grafana_promtail_loki_demo_loki

grafana_promtail_loki_demo_promtail

可查看loki & promtail 启动失败报错。

因此可以单独启动某个 service, 命令如下:

#docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up  <service name>  <-d>

Note:

1. -d : 后台运行

2. service name 通过 docker-compose-grafana-promtail-loki.yml 中

   services :

   loki:

      ....

  promtail:

       ...

  granfana:

     ...

# Git Bash 查询 docker-compose-grafana-promtail-loki.yml 结果如下:

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Soft-dev/Dokcer/docker-compose-master/Windows/grafana_promtail_loki
$ ls
docker-compose-grafana-promtail-loki.yml  grafana_promtail_loki/  run.md

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Soft-dev/Dokcer/docker-compose-master/Windows/grafana_promtail_loki
$ cat docker-compose-grafana-promtail-loki.yml
# docker-compose配置参考:https://github.com/grafana/loki/blob/master/production/docker-compose.yaml
version: "3"

# 网桥loki -> 方便相互通讯
networks:
  loki:

services:
  # Loki日志系统
  loki:
    image: grafana/loki:latest
    container_name: grafana_promtail_loki_demo_loki
    restart: unless-stopped
    ports:
      - "3100:3100"
    volumes:
      - ./grafana_promtail_loki/loki:/etc/loki
      - ./grafana_promtail_loki/wal:/wal
    command: -config.file=/etc/loki/loki-local-config.yaml
    networks:
      - loki

  # Loki官方支持的日志采集端
  promtail:
    image: grafana/promtail:latest
    container_name: grafana_promtail_loki_demo_promtail
    restart: unless-stopped
    volumes:
      - ./grafana_promtail_loki/logs:/var/logs
      - ./grafana_promtail_loki/promtail:/etc/promtail
    command: -config.file=/etc/promtail/promtail-docker-config.yaml
    networks:
      - loki

  # 用于UI展示
  grafana:
    image: grafana/grafana:latest
    container_name: grafana_promtail_loki_demo_grafana
    restart: unless-stopped
    ports:
      - "3000:3000"
    volumes:
      - "./grafana_promtail_loki/grafana/data:/var/lib/grafana"
      #      - "./grafana_promtail_loki/grafana/config:/etc/grafana"
      - "./grafana_promtail_loki/grafana/log:/var/log/grafana"
    environment:
      GF_EXPLORE_ENABLED: "true"
    networks:
      - loki

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Soft-dev/Dokcer/docker-compose-master/Windows/grafana_promtail_loki
$ 
''' 
1. 切换路径到 D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki\
2 . 启动某个service
    #前端运行
    # docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up promtail

'''


D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up promtail
[+] Running 2/2
 - Network grafana_promtail_loki_loki             Created                                                                                                                                                    0.8s
 - Container grafana_promtail_loki_demo_promtail  Created                                                                                                                                                    0.2s
Attaching to grafana_promtail_loki_demo_promtail
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:10.201691148Z caller=promtail.go:123 msg="Reloading configuration file" md5sum=afd7ad07c892f8a059abf51b346cafe0
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:10.205267637Z caller=server.go:323 http=[::]:37235 grpc=[::]:37881 msg="server listening on addresses"
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:10.20647781Z caller=main.go:171 msg="Starting Promtail" version="(version=2.7.1, branch=HEAD, revision=e0af1cc8a)"
grafana_promtail_loki_demo_promtail  | level=warn ts=2023-01-06T05:57:10.206547905Z caller=promtail.go:220 msg="enable watchConfig"
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:15.204728147Z caller=filetargetmanager.go:352 msg="Adding target" key="/var/logs/*log:{app=\"varlogs\", job=\"varlogs\"}"
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:15.208100131Z caller=filetarget.go:282 msg="watching new directory" directory=/var/logs
grafana_promtail_loki_demo_promtail  | level=info ts=2023-01-06T05:57:15.209740792Z caller=tailer.go:143 component=tailer msg="tail routine: started" path=/var/logs/app.log
grafana_promtail_loki_demo_promtail  | ts=2023-01-06T05:57:15.209785354Z caller=log.go:168 level=info msg="Seeked /var/logs/app.log - &{Offset:8621 Whence:0}"
[+] Running 1/1ping... (press Ctrl+C again to force)
 - Container grafana_promtail_loki_demo_promtail  Stopped                                                                                                                                                    0.4s
canceled
'''
    # 后台运行
    # docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up promtail -d

'''

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up promtail -d
[+] Running 1/1
 - Container grafana_promtail_loki_demo_promtail  Started                                                                                                                                                    1.0s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>

遇到问题如下:

1. 单独启动loki,报错如下所示:

d:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up  loki
[+] Running 1/0
 - Container grafana_promtail_loki_demo_loki  Created                                                                                                   0.0s
Attaching to grafana_promtail_loki_demo_loki
grafana_promtail_loki_demo_loki  | creating WAL folder at "/wal": mkdir wal: permission denied
grafana_promtail_loki_demo_loki  | error initialising module: ingester
grafana_promtail_loki_demo_loki  | github.com/grafana/dskit/modules.(*Manager).initModule

d:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up  loki
[+] Running 1/0
 - Container grafana_promtail_loki_demo_loki  Created                                                                                                   0.0s
Attaching to grafana_promtail_loki_demo_loki
grafana_promtail_loki_demo_loki  | creating WAL folder at "/wal": mkdir wal: permission denied
grafana_promtail_loki_demo_loki  | error initialising module: ingester
grafana_promtail_loki_demo_loki  | github.com/grafana/dskit/modules.(*Manager).initModule
grafana_promtail_loki_demo_loki  |      /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:122
grafana_promtail_loki_demo_loki  | github.com/grafana/dskit/modules.(*Manager).InitModuleServices
grafana_promtail_loki_demo_loki  |      /src/loki/vendor/github.com/grafana/dskit/modules/modules.go:92
grafana_promtail_loki_demo_loki  | github.com/grafana/loki/pkg/loki.(*Loki).Run
grafana_promtail_loki_demo_loki  |      /src/loki/pkg/loki/loki.go:363
grafana_promtail_loki_demo_loki  | main.main

解决方法:

为loki 服务添加:- ./grafana_promtail_loki/wal:/wal 

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Soft-dev/Dokcer/docker-compose-master/Windows/grafana_promtail_loki
$ cat docker-compose-grafana-promtail-loki.yml
# docker-compose配置参考:https://github.com/grafana/loki/blob/master/production/docker-compose.yaml
version: "3"

# 网桥loki -> 方便相互通讯
networks:
  loki:

services:
  # Loki日志系统
  loki:
    image: grafana/loki:latest
    container_name: grafana_promtail_loki_demo_loki
    restart: unless-stopped
    ports:
      - "3100:3100"
    volumes:
      - ./grafana_promtail_loki/loki:/etc/loki
      - ./grafana_promtail_loki/wal:/wal     # 添加
    command: -config.file=/etc/loki/loki-local-config.yaml
    networks:
      - loki

  # Loki官方支持的日志采集端
 

docker搭建Grafana+Loki+Promtail日志分析

如果docker-compose 一起启动多个services 有错,可单独启动某个services,联调无误后,以后可以通过docker-compose 一次性启动。

1. # CMD 管理员权限
2. # 下载的docker-compose-master.zip 解压路径为:d:\Soft\Soft-dev\Dokcer\

3. # 启动命令如下:
docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up -d

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up -d
[+] Running 4/4
 - Network grafana_promtail_loki_loki             Created                                                                                                                                                    0.7s
 - Container grafana_promtail_loki_demo_promtail  Started                                                                                                                                                    1.6s
 - Container grafana_promtail_loki_demo_grafana   Started                                                                                                                                                    3.2s
 - Container grafana_promtail_loki_demo_loki      Started                                                                                                                                                    2.7s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>



4. # docker-compose 查询运行 serives
docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                    STATUS              CONFIG FILES
grafana_promtail_loki   running(3)          D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki\docker-compose-grafana-promtail-loki.yml

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>




5. # docker 查询已运行 serives
docker ps

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED          STATUS          PORTS                    NAMES
6f692c924ce1   grafana/grafana:latest          "/run.sh"                22 seconds ago   Up 18 seconds   0.0.0.0:3000->3000/tcp   grafana_promtail_loki_demo_grafana
369d843c4cbb   grafana/loki:latest             "/usr/bin/loki -conf…"   22 seconds ago   Up 18 seconds   0.0.0.0:3100->3100/tcp   grafana_promtail_loki_demo_loki
2393ebad644f   grafana/promtail:latest         "/usr/bin/promtail -…"   22 seconds ago   Up 19 seconds                            grafana_promtail_loki_demo_promtail
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   16 hours ago     Up 16 hours     0.0.0.0:80->80/tcp       vector03

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>




6. # docker-compose 关闭所有serives
docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki down

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki down
[+] Running 4/4
 - Container grafana_promtail_loki_demo_promtail  Removed                                                                                                                                                    0.8s
 - Container grafana_promtail_loki_demo_grafana   Removed                                                                                                                                                    0.6s
 - Container grafana_promtail_loki_demo_loki      Removed                                                                                                                                                    6.0s
 - Network grafana_promtail_loki_loki             Removed                                                                                                                                                    0.7s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>




7. # 再次查询已运行 serives

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                STATUS              CONFIG FILES

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>


D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS        PORTS                NAMES
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   16 hours ago   Up 16 hours   0.0.0.0:80->80/tcp   vector03

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>

# 查询服务

方法一:Docker Desktop

方法二: 命令

# docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls

或者

# docker ps

1. # CMD 管理员权限

2. # 启动服务器查询 (无服务)

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                STATUS              CONFIG FILES

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>


3. # 单独 后台运行启动 服务loki

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki up loki -d
[+] Running 2/2
 - Network grafana_promtail_loki_loki         Created                                                                                                                                                        0.8s
 - Container grafana_promtail_loki_demo_loki  Started                                                                                                                                                        1.2s

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>


4. # 启动服务器查询 (1服务:loki)

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>docker-compose -f docker-compose-grafana-promtail-loki.yml -p grafana_promtail_loki ls
NAME                    STATUS              CONFIG FILES
grafana_promtail_loki   running(1)          D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki\docker-compose-grafana-promtail-loki.yml

D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>
D:\Soft\Soft-dev\Dokcer\docker-compose-master\Windows\grafana_promtail_loki>

3. 通过docker-compose 安装Grafana+loki+Promtail

4. 查看

查看 运行 容器

方法一:命令行查询

# CMD 管理员 or GitBash

#docker ps 


C:\Windows\system32>docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS        PORTS                    NAMES
c14308ada3b1   grafana/loki:latest             "/usr/bin/loki -conf…"   10 hours ago   Up 10 hours   0.0.0.0:3100->3100/tcp   grafana_promtail_loki_demo_loki
c6e8d3289c51   grafana/promtail:latest         "/usr/bin/promtail -…"   11 hours ago   Up 10 hours                            grafana_promtail_loki_demo_promtail
a8065ef7019b   grafana/grafana:latest          "/run.sh"                11 hours ago   Up 10 hours   0.0.0.0:3000->3000/tcp   grafana_promtail_loki_demo_grafana
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   12 hours ago   Up 12 hours   0.0.0.0:80->80/tcp       vector03

C:\Windows\system32>

查看 运行 容器

方法二:Docker Desktop

docker搭建Grafana+Loki+Promtail日志分析

查询所有容器:

# CMD 管理员
# docker ps -a


C:\Windows\system32>
C:\Windows\system32>docker ps -a
CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS                    PORTS                    NAMES
c14308ada3b1   grafana/loki:latest             "/usr/bin/loki -conf…"   10 hours ago   Up 10 hours               0.0.0.0:3100->3100/tcp   grafana_promtail_loki_demo_loki
c6e8d3289c51   grafana/promtail:latest         "/usr/bin/promtail -…"   11 hours ago   Up 10 hours                                        grafana_promtail_loki_demo_promtail
a8065ef7019b   grafana/grafana:latest          "/run.sh"                11 hours ago   Up 10 hours               0.0.0.0:3000->3000/tcp   grafana_promtail_loki_demo_grafana
2f973e49c673   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   12 hours ago   Up 12 hours               0.0.0.0:80->80/tcp       vector03
c6cf31e5ad58   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   17 hours ago   Exited (0) 17 hours ago                            vector01
abbd527ebd3d   timberio/vector:0.10.0-alpine   "/usr/local/bin/vect…"   17 hours ago   Created                                            vector
9066857121a2   grafana/grafana                 "/run.sh"                23 hours ago   Exited (0) 11 hours ago                            grafana
16218564a475   hello-world                     "/hello"                 24 hours ago   Exited (0) 24 hours ago                            reverent_jemison

C:\Windows\system32>

配置Grafana

1. 浏览器打开 grafana 控制台,并进入控制台

浏览器访问:

http://localhost:3000  or  http://127.0.0.1:3000

 username& pwd: admin\admin

默认账号和密码都是 admin ,进入了以后 跳过修改密码

docker搭建Grafana+Loki+Promtail日志分析

 2. 设置 grafana 的数据源为 loki

docker搭建Grafana+Loki+Promtail日志分析

 docker搭建Grafana+Loki+Promtail日志分析

 3. 配置数据源Loki 信息

docker搭建Grafana+Loki+Promtail日志分析

 4. Vector

#  GitBash or cmd 管理员

创建container

# docker run -d -p 80:80 --name vector timberio/vector:0.10.0-alpine

magx@DESKTOP-C1MJB99 MINGW64 /d/Soft/Docker
$ docker run -d -p 80:80 --name vector timberio/vector:0.10.0-alpine
aab569a25ead1cb385edaf22144ec8cc46b4e956b4c766051cf4deea47a28d01
 

有时候需要Docker修改容器内部文件,方法多种,推荐 使用-v挂载文件夹(推荐)

方法是在启动的时候使用-v将容器内部的文件夹挂载(映射)到本地的某个路径下,以后以后可以直接在本地修改,不需要进入容器内部.

#冒号前是本地路径(需要绝对路径),冒号后是容器中的路径文章来源地址https://www.toymoban.com/news/detail-488933.html

到了这里,关于docker搭建Grafana+Loki+Promtail日志分析的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • Loki+Promtail+Grafana 监控 K8s 日志

    Loki 架构: 1、loki:服务端,负责存储日志和处理查询 2、promtail:采集端,负责采集日志发送给loki 3、grafana:负责采集日志的展示 创建 yaml 文件 cat loki-rbac.yaml cat loki-nodeport.yaml cat loki.yaml cat loki-configmap.yaml cat loki-promtail-rbac.yaml cat loki-promtail.yaml cat loki-promtail-configmap.yaml 2、创

    2024年02月16日
    浏览(35)
  • Loki+Grafana监控docker容器日志

    目标:最近开发人员时常需要查看各个环境项目中容器日志,而直接通过ssh终端使用docker logs命令查看日志不太安全,这会导致很多人员知道服务器的账户和密码,有没有一种可以收集所有容器日志的平台系统。那么这套系统就是Loki+Grafana组合。 重要提醒 :不要用于生产环境

    2024年02月12日
    浏览(33)
  • prometheus|云原生|轻型日志收集系统loki+promtail的部署说明

    日志------ 每一个程序,服务都应该有保留日志,日志的作用第一是记录程序运行的情况,在出错的时候能够记录错误情况,简单来说就是审计工作,例如nginx服务的日志,kubernetes集群的pod运行日志,Linux系统的系统日志。 其次是数据库的日志,记录数据库的运行细节,在需要

    2024年02月04日
    浏览(28)
  • Rancher中使用promtail+loki+grafna收集k8s日志并展示

    根据应用需求和日志数量级别选择对应的日志收集、过滤和展示方式,当日志量不太大,又想简单集中管理查看日志时,可使用promtail+loki+grafna的方式。本文找那个loki和grafana外置在了k8s集群之外。 方式一: 方式二: 登录rancher,选择集群→应用→Chart仓库→创建,配置仓库

    2024年01月20日
    浏览(30)
  • Grafana Loki 日志传输工具

    有各种不同的工具支持传输日志到Loki,有Grafana开发和第三方开发的工具。 Grafana 开发的工具: Grafana Agent Grafna 技术栈推荐客户端,支持收集度量、日志、跟踪和持续性能分析的遥测数据,跟Prometheus、OpenTelemetry、Grafana开源生态系统完全兼容 Promtail K8s首选的客户端,能够自动

    2024年02月02日
    浏览(26)
  • Grafana系列-Loki-基于日志实现告警

    Loki 系列文章 实际应用中除了基于 Metrics 告警, 往往还有基于日志的告警需求, 可以作为基于 Metrics 告警之外的一个补充. 典型如基于 NGINX 日志的错误率告警.本文将介绍如何基于 Loki 实现基于日志的告警. 本文我们基于以下 2 类实际场景进行实战演练: 基于 NGINX 日志的错误率告

    2024年02月05日
    浏览(29)
  • k8s日志收集组件 Grafana loki --- 理论篇

    当我们在k8s上运行程序时,习惯的会使用ELK来收集和查询程序运行日志。今天我们介绍一款新的专为日志收集而生的神器:Grafana loki。Grafana Loki 是一组组件,可以组合成一个功能齐全的日志堆栈。 与其他日志记录系统不同,Loki 仅构建索引有关日志的元数据:标签(就像 P

    2024年02月10日
    浏览(37)
  • logstack 日志技术栈-04-opensource 开源工具 OpenObserve+Grafana Loki

    日志管理包含日志数据存储、处理、分析和可视化,通过利用日志管理工具,可以监控性能趋势、解决问题、检测异常并优化整体系统性能。 近年来,开源日志管理解决方案在大家寻求灵活且经济有效的方式来管理现代系统典型的大量日志数据时,获得了显著的关注。这些工

    2024年01月21日
    浏览(36)
  • 使用kubeshpere创建k8s容器日志系统grafana-loki

    k8s日志聚合平台grafana-loki,可以统一查看所有容器的日志运行。 效果:  使用kubeshpere创建loki应用十分的方便,减少了很多操作,易用的同时也实用,k8s不再是精通人员才能使用的编排系统。 1、在企业空间中,添加bitnami应用仓库 https://charts.bitnami.com/bitnami  2、创建loki应用

    2024年02月12日
    浏览(38)
  • grafana呈现loki从k8s/ingress-nginx中收集的日志-地区经纬度部分

    1,在用grafana展示loki日志的时候,想到用可视化的方式来呈现更直观,于是网上查找,找到一篇《如何用Loki来绘制Ingress Nginx监控大屏》被复用多次,按照其过程实践引用了12559的面板,但效果与原文描述的不一致,可能是gf版本,或者插件版本变更等原因造成。尝试了几次终

    2024年02月21日
    浏览(38)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包