Observability：为 Logstash 输出配置 SSL/TLS - Elastic Agent

​

./bin/elasticsearch-certutil ca --pem

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ ./bin/elasticsearch-certutil ca --pem
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.zip]: 
$ ls
LICENSE.txt          bin                  elastic-stack-ca.zip logs
NOTICE.txt           config               jdk.app              modules
README.asciidoc      data                 lib                  plugins
$ unzip elastic-stack-ca.zip 
Archive:  elastic-stack-ca.zip
   creating: ca/
  inflating: ca/ca.crt               
  inflating: ca/ca.key    

./bin/elasticsearch-certutil cert \
  --name client \
  --ca-cert /path/to/ca/ca.crt \
  --ca-key /path/to/ca/ca.key \
  --pem

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ ./bin/elasticsearch-certutil cert \
>   --name client \
>   --ca-cert ./ca/ca.crt \
>   --ca-key ./ca/ca.key \
>   --pem
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.


    * All certificates generated by this tool will be signed by a certificate authority (CA)
      unless the --self-signed command line option is specified.
      The tool can automatically generate a new CA for you, or you can provide your own with
      the --ca or --ca-cert command line options.


By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Please enter the desired output file [certificate-bundle.zip]: 

Certificates written to /Users/liuxg/elastic/elasticsearch-8.8.1/certificate-bundle.zip

This file should be properly secured as it contains the private key for 
your instance.
After unzipping the file, there will be a directory for each instance.
Each instance has a certificate and private key.
For each Elastic product that you wish to configure, you should copy
the certificate, key, and CA certificate to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
$ unzip certificate-bundle.zip 
Archive:  certificate-bundle.zip
   creating: client/
  inflating: client/client.crt       
  inflating: client/client.key     

./bin/elasticsearch-certutil cert \
  --name logstash \
  --ca-cert /path/to/ca/ca.crt \
  --ca-key /path/to/ca/ca.key \
  --dns your.host.name.here \
  --ip 192.0.2.1 \
  --pem

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ ls
LICENSE.txt            ca                     data                   logs
NOTICE.txt             certificate-bundle.zip elastic-stack-ca.zip   modules
README.asciidoc        client                 jdk.app                plugins
bin                    config                 lib
$ rm certificate-bundle.zip 
remove certificate-bundle.zip? y
$ ./bin/elasticsearch-certutil cert \
>   --name logstash \
>   --ca-cert ./ca/ca.crt \
>   --ca-key ./ca/ca.key \
>   --dns ubuntu2004 \
>   --ip 192.168.0.8 \
>   --pem
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.


    * All certificates generated by this tool will be signed by a certificate authority (CA)
      unless the --self-signed command line option is specified.
      The tool can automatically generate a new CA for you, or you can provide your own with
      the --ca or --ca-cert command line options.


By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Please enter the desired output file [certificate-bundle.zip]: 

Certificates written to /Users/liuxg/elastic/elasticsearch-8.8.1/certificate-bundle.zip

This file should be properly secured as it contains the private key for 
your instance.
After unzipping the file, there will be a directory for each instance.
Each instance has a certificate and private key.
For each Elastic product that you wish to configure, you should copy
the certificate, key, and CA certificate to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.
$ unzip certificate-bundle.zip 
Archive:  certificate-bundle.zip
   creating: logstash/
  inflating: logstash/logstash.crt   
  inflating: logstash/logstash.key  

openssl pkcs8 -inform PEM -in logstash.key -topk8 -nocrypt -outform PEM -out logstash.pkcs8.key

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ cd logstash/
$ openssl pkcs8 -inform PEM -in logstash.key -topk8 -nocrypt -outform PEM -out logstash.pkcs8.key
$ ls
logstash.crt       logstash.key       logstash.pkcs8.key

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1/logstash
$ scp logstash.crt parallels@ubuntu2004:/home/parallels/logstash/logstash-8.8.1/certs
logstash.crt                                                 100% 1188     3.0MB/s   00:00    
$ scp logstash.pkcs8.key parallels@ubuntu2004:/home/parallels/logstash/logstash-8.8.1/certs
logstash.pkcs8.key                                           100% 1708     1.2MB/s   00:00  

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ cd ca/
$ ls
ca.crt ca.key
$ scp ca.crt parallels@ubuntu2004:/home/parallels/logstash/logstash-8.8.1/certs
ca.crt          

parallels@ubuntu2004:~/logstash/logstash-8.8.1/certs$ pwd
/home/parallels/logstash/logstash-8.8.1/certs
parallels@ubuntu2004:~/logstash/logstash-8.8.1/certs$ ls
ca.crt  logstash.crt  logstash.pkcs8.key

$ pwd
/Users/liuxg/elastic/elasticsearch-8.8.1
$ cd ca/
$ ls
ca.crt ca.key
$ scp ca.crt parallels@ubuntu2004:/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs
ca.crt                                                       100% 1200   890.5KB/s   00:00    
$ cd ..
$ cd client/
$ ls
client.crt client.key
$ scp client.crt parallels@ubuntu2004:/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs
client.crt                                                   100% 1143   873.4KB/s   00:00    
$ scp client.key parallels@ubuntu2004:/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs
client.key                                                   100% 1675     1.2MB/s   00:00  

parallels@ubuntu2004:~/fleet/elastic-agent-8.8.1-linux-arm64/certs$ pwd
/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs
parallels@ubuntu2004:~/fleet/elastic-agent-8.8.1-linux-arm64/certs$ ls
ca.crt  client.crt  client.key  

- pipeline.id: elastic-agent-pipeline
  path.config: "/etc/path/to/elastic-agent-pipeline.conf"

input {
  elastic_agent {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/home/parallels/logstash/logstash-8.8.1/certs/ca.crt"]
    ssl_certificate => "/home/parallels/logstash/logstash-8.8.1/certs/logstash.crt"
    ssl_key => "/home/parallels/logstash/logstash-8.8.1/certs/logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
   stdout {}

   elasticsearch {
      hosts => ["https://192.168.0.3:9200"]
      index => "data-%{+YYYY.MM.dd}"
      ssl => true
      ilm_enabled => true
      user => "elastic"
      password => "z5nxTriCD4fi7jSS=GFM"
      ca_trusted_fingerprint => "783663875df7ae1daf3541ab293d8cd48c068b3dbc2d9dd6fa8a668289986ac2"
    }
}

./bin/logstash -f logstash.conf

outputs:
  default:
    type: logstash
    hosts: ["192.168.0.8:5044"]
    ssl.enabled: true
    ssl.certificate: "/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs/client.crt"
    ssl.key: "/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs/client.key"
    ssl.certificate_authorities: ["/home/parallels/fleet/elastic-agent-8.8.1-linux-arm64/certs/ca.crt"]
    # ssl.verification _mode: "none"