win10安装FileBeat

 

  前面我们安装了ELK(参见win10安装ELK),数据流向是:L -> E -> K,其实L的前面还可以再接一根管道B。这个B就是Beat。Beat组件的加入,打破了ELK的三国鼎立,ELK成了Elastic Stack。有各种Beat可以成为Logstash或Elasticsearch的数据源:FileBeat、PacketBeat和MetricBeat。对日志文件的传输,首选FileBeat。FileBeat可以对接Logstash,也可以直接对接Elasticsearch。

  首先,去下载地址https://www.elastic.co/cn/downloads/beats/filebeat下载压缩包,我们还是以最新的7.9.0版本为例:

win10安装FileBeat

 

   我们选择windows版本64位的压缩包,下载后解压到D盘elk目录下:

win10安装FileBeat

 

  点击开始菜单 -> 找到W开头的菜单项 -> 点开Windows PowerShell -> 右键点击Winows PowerShell(x86) -> 选择“以管理员身份运行”:

win10安装FileBeat

  

   进入FileBeat安装目录,执行安装命令,不出意外,会报错:

PS C:\Users\wulf> cd D:\elk\filebeat-7.9.0-windows-x86_64PS D:\elk\filebeat-7.9.0-windows-x86_64> .\install-service-filebeat.ps1
.\install-service-filebeat.ps1 : 无法加载文件 D:\elk\filebeat-7.9.0-windows-x86_64\install-service-filebeat.ps1,因为在
此系统上禁止运行脚本。有关详细信息,请参阅 https:/go.microsoft.com/fwlink/?LinkID=135170 中的 about_Execution_Policies
。
所在位置 行:1 字符: 1
+ .\install-service-filebeat.ps1
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [],PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess
PS D:\elk\filebeat-7.9.0-windows-x86_64>

  

  为啥呢?虽然我们用管理员身份运行PowerShell,但执行的命令有问题,我们需要换种方式执行以上命令:

PS D:\elk\filebeat-7.9.0-windows-x86_64> Get-ExecutionPolicy
Restricted
PS D:\elk\filebeat-7.9.0-windows-x86_64> Set-ExecutionPolicy UnRestricted

执行策略更改
执行策略可帮助你防止执行不信任的脚本。更改执行策略可能会产生安全风险,如 https:/go.microsoft.com/fwlink/?LinkID=135170
中的 about_Execution_Policies 帮助主题所述。是否要更改执行策略?
[Y] 是(Y)  [A] 全是(A)  [N] 否(N)  [L] 全否(L)  [S] 暂停(S)  [?] 帮助 (默认值为“N”): y
PS D:\elk\filebeat-7.9.0-windows-x86_64> .\install-service-filebeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  filebeat           filebeat


PS D:\elk\filebeat-7.9.0-windows-x86_64>

 

  从上面可以看到,是执行策略阻止了我们对fileBeat的安装,变更一下就好了。安装好后PowerShell窗口就可以关掉了。

    接着修改配置文件,进入D:\elk\filebeat-7.9.0-windows-x86_64目录,复制filebeat.yml,重命名为filebeat-simple.yml,修改它,内容如下:

 

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - D:\\wlf\\logs\\hello*.log

output.logstash:
  hosts: ["localhost:5044"]

  这里指定FileBeat读取D盘logs目录下一个叫hello.log的日志文件。 

  先把ELK启起来。记得启动顺序:E -> K -> L,其中L启动时指定配置文件:logstash-simple.conf,把原来的输入源由stdin改为FileBeat:

input{
 beats {
    port => "5044"
  }
}

output {
  stdout { codec => rubydebug }
}

  

  logstash启动日志:

C:\Users\wulf>D:

D:\>cd elk\logstash-7.9.0\bin

D:\elk\logstash-7.9.0\bin>.\logstash -f ..\config\logstash-simple.conf
Sending Logstash logs to D:/elk/logstash-7.9.0/logs which is now configured via log4j2.properties
[2020-09-03T21:57:41,911][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.9.0", "jruby.version"=>"jruby 9.2.12.0 (2.5.7) 2020-07-01 db01a49ba6 Java HotSpot(TM) 64-Bit Server VM 25.102-b14 on 1.8.0_102-b14 +indy +jit [mswin32-x86_64]"}
[2020-09-03T21:57:42,259][WARN ][logstash.config.source.multilocal] Ignoring the pipelines.yml file because modules or command line options are specified
[2020-09-03T21:57:44,742][INFO ][org.reflections.Reflections] Reflections took 46 ms to scan 1 urls, producing 22 keys and 45 values
[2020-09-03T21:57:44,998][WARN ][org.logstash.netty.SslContextBuilder] JCE Unlimited Strength Jurisdiction Policy not installed - max key length is 128 bits
[2020-09-03T21:57:47,363][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>500, "pipeline.sources"=>["D:/elk/logstash-7.9.0/config/logstash-simple.conf"], :thread=>"#<Thread:0x30aae30d run>"}
[2020-09-03T21:57:48,518][INFO ][logstash.javapipeline    ][main] Pipeline Java execution initialization time {"seconds"=>1.14}
[2020-09-03T21:57:48,547][INFO ][logstash.inputs.beats    ][main] Beats inputs: Starting input listener {:address=>"0.0.0.0:5044"}
[2020-09-03T21:57:48,573][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
[2020-09-03T21:57:48,702][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2020-09-03T21:57:48,820][INFO ][org.logstash.beats.Server][main][af3dcc0a25640c2afc7ea292b455b1260403e81008f9a1579f987486d2f7e56b] Starting server on port: 5044
[2020-09-03T21:57:49,215][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

 

  用一个定时任务每5秒往D:\wlf\log目录下的hello-2020-09-03.0.log文件打印日志:

package com.wlf.elasticsearchstatictis;

import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;

@Slf4j
@SpringBootApplication
@EnableScheduling
public class Begin {
    public static void main(String[] args) {
        SpringApplication.run(Begin.class, args);
    }

    @Scheduled(fixedRate = 5000)
    public void logProduceTask()
    {
        log.info("hello, world.");
    }
}

 

  然后启动FileBeat:

C:\Users\wulf>d:

D:\>cd elk\filebeat-7.9.0-windows-x86_64

D:\elk\filebeat-7.9.0-windows-x86_64>.\filebeat -e -c filebeat-simple.yml
2020-09-03T22:02:33.203+0800    INFO    instance/beat.go:640    Home path: [D:\elk\filebeat-7.9.0-windows-x86_64] Config path: [D:\elk\filebeat-7.9.0-windows-x86_64] Data path: [D:\elk\filebeat-7.9.0-windows-x86_64\data] Logs path: [D:\elk\filebeat-7.9.0-windows-x86_64\logs]
2020-09-03T22:02:33.206+0800    INFO    instance/beat.go:648    Beat ID: ae375dc0-d6e2-488c-be87-2544c05b1242
2020-09-03T22:02:33.209+0800    INFO    [beat]  instance/beat.go:976    Beat info       {"system_info": {"beat": {"path": {"config": "D:\\elk\\filebeat-7.9.0-windows-x86_64", "data": "D:\\elk\\filebeat-7.9.0-windows-x86_64\\data", "home": "D:\\elk\\filebeat-7.9.0-windows-x86_64", "logs": "D:\\elk\\filebeat-7.9.0-windows-x86_64\\logs"}, "type": "filebeat", "uuid": "ae375dc0-d6e2-488c-be87-2544c05b1242"}}}
2020-09-03T22:02:33.211+0800    INFO    [beat]  instance/beat.go:985    Build info      {"system_info": {"build": {"commit": "b2ee705fc4a59c023136c046803b56bc82a16c8d", "libbeat": "7.9.0", "time": "2020-08-11T20:11:10.000Z", "version": "7.9.0"}}}
2020-09-03T22:02:33.211+0800    INFO    [beat]  instance/beat.go:988    Go runtime info {"system_info": {"go": {"os":"windows","arch":"amd64","max_procs":4,"version":"go1.14.4"}}}
2020-09-03T22:02:33.279+0800    INFO    [beat]  instance/beat.go:992    Host info       {"system_info": {"host": {"architecture":"x86_64","boot_time":"2020-08-19T01:16:40.99+08:00","name":"wulf00","ip":["fe80::8d8e:da9f:cdde:a6b8/64","2.0.2.177/24","fe80::589d:d728:5523:99e5/64","10.73.166.158/24","fe80::759e:b0eb:609:cf8f/64","169.254.207.143/16","fe80::f58b:cdd3:6144:9492/64","169.254.148.146/16","fe80::b4c3:3952:c602:bbb6/64","10.129.217.84/21","fe80::cbd:73cc:2721:24a0/64","169.254.36.160/16","::1/128","127.0.0.1/8"],"kernel_version":"10.0.18362.1016 (WinBuild.160101.0800)","mac":["00:ff:ef:08:d8:e5","54:e1:ad:57:79:63","a0:af:bd:73:a2:09","a2:af:bd:73:a2:08","a0:af:bd:73:a2:08","00:ff:5e:c9:2d:c6"],"os":{"family":"windows","platform":"windows","name":"Windows 10 Pro","version":"10.0","major":10,"minor":0,"patch":0,"build":"18362.1016"},"timezone":"CST","timezone_offset_sec":28800,"id":"bd5672aa-84f8-4043-b25f-47453b5a9362"}}}
2020-09-03T22:02:33.280+0800    INFO    [beat]  instance/beat.go:1021   Process info    {"system_info": {"process": {"cwd": "D:\\elk\\filebeat-7.9.0-windows-x86_64", "exe": "D:\\elk\\filebeat-7.9.0-windows-x86_64\\filebeat.exe", "name": "filebeat.exe", "pid": 68892, "ppid": 68040, "start_time": "2020-09-03T22:02:30.172+0800"}}}
2020-09-03T22:02:33.280+0800    INFO    instance/beat.go:299    Setup Beat: filebeat; Version: 7.9.0
2020-09-03T22:02:33.294+0800    INFO    [publisher]     pipeline/module.go:113  Beat name: wulf00
2020-09-03T22:02:33.302+0800    WARN    beater/filebeat.go:178  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-09-03T22:02:33.303+0800    INFO    instance/beat.go:450    filebeat start running.
2020-09-03T22:02:33.303+0800    INFO    [monitoring]    log/log.go:118  Starting metrics logging every 30s
2020-09-03T22:02:33.309+0800    INFO    memlog/store.go:119     Loading data file of D:\elk\filebeat-7.9.0-windows-x86_64\data\registry\filebeat succeeded. Active transaction id=0
2020-09-03T22:02:33.327+0800    INFO    memlog/store.go:124     Finished loading transaction log file for D:\elk\filebeat-7.9.0-windows-x86_64\data\registry\filebeat. Active transaction id=427
2020-09-03T22:02:33.327+0800    WARN    beater/filebeat.go:381  Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.
2020-09-03T22:02:33.333+0800    INFO    [registrar]     registrar/registrar.go:108      States Loaded from registrar: 4
2020-09-03T22:02:33.334+0800    INFO    [crawler]       beater/crawler.go:71    Loading Inputs: 1
2020-09-03T22:02:33.337+0800    INFO    log/input.go:157        Configured paths: [D:\wlf\logs\hello*.log]
2020-09-03T22:02:33.337+0800    INFO    [crawler]       beater/crawler.go:141   Starting input (ID: 9386287014943630624)
2020-09-03T22:02:33.339+0800    INFO    [crawler]       beater/crawler.go:108   Loading and starting Inputs completed. Enabled inputs: 1
2020-09-03T22:03:03.319+0800    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":281,"time":{"ms":281}},"total":{"ticks":437,"time":{"ms":437},"value":437},"user":{"ticks":156,"time":{"ms":156}}},"handles":{"open":213},"info":{"ephemeral_id":"09af0d14-6589-4eeb-8fd1-3315aba33f07","uptime":{"ms":32876}},"memstats":{"gc_next":16354592,"memory_alloc":8873952,"memory_total":41138008,"rss":48308224},"runtime":{"goroutines":23}},"filebeat":{"events":{"added":1,"done":1},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"logstash"},"pipeline":{"clients":1,"events":{"active":0,"filtered":1,"total":1}}},"registrar":{"states":{"current":4,"update":1},"writes":{"success":1,"total":1}},"system":{"cpu":{"cores":4}}}}}
2020-09-03T22:03:33.309+0800    INFO    [monitoring]    log/log.go:145  Non-zero metrics in the last 30s        {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":281},"total":{"ticks":437,"value":437},"user":{"ticks":156}},"handles":{"open":210},"info":{"ephemeral_id":"09af0d14-6589-4eeb-8fd1-3315aba33f07","uptime":{"ms":62867}},"memstats":{"gc_next":16354592,"memory_alloc":8953176,"memory_total":41217232,"rss":-24576},"runtime":{"goroutines":23}},"filebeat":{"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":4}}}}}
2020-09-03T22:03:33.395+0800    INFO    log/harvester.go:297    Harvester started for file: D:\wlf\logs\hello-2020-09-03.0.log
2020-09-03T22:03:34.405+0800    INFO    [publisher_pipeline_output]     pipeline/output.go:143  Connecting to backoff(async(tcp://localhost:5044))
2020-09-03T22:03:34.405+0800    INFO    [publisher]     pipeline/retry.go:219   retryer: send unwait signal to consumer
2020-09-03T22:03:34.417+0800    INFO    [publisher]     pipeline/retry.go:223     done
2020-09-03T22:03:34.463+0800    INFO    [publisher_pipeline_output]     pipeline/output.go:151  Connection to backoff(async(tcp://localhost:5044)) established

 

  代码跑,日志刷,logstash也在刷:

{
      "@version" => "1",
         "agent" => {
                  "id" => "ae375dc0-d6e2-488c-be87-2544c05b1242",
                "type" => "filebeat",
                "name" => "wulf00",
             "version" => "7.9.0",
        "ephemeral_id" => "09af0d14-6589-4eeb-8fd1-3315aba33f07",
            "hostname" => "wulf00"
    },
       "message" => "22:03:23.916 [main] [] [] INFO  com.wlf.elasticsearchstatictis.Begin - No active profile set, falling back to default profiles: default",
           "log" => {
        "offset" => 20893,
          "file" => {
            "path" => "D:\\wlf\\logs\\hello-2020-09-03.0.log"
        }
    },
         "input" => {
        "type" => "log"
    },
          "tags" => [
        [0] "beats_input_codec_plain_applied"
    ],
          "host" => {
        "name" => "wulf00"
    },
           "ecs" => {
        "version" => "1.5.0"
    },
    "@timestamp" => 2020-09-03T14:03:33.404Z
}

 

win10安装FileBeat

 

 

 

 

win10安装FileBeat

上一篇:C#中使用swagger小技巧


下一篇:C#时间与String转换