Advanced Log Collection and Shipping :
Lab 1 : Centralized Log Collection & Indexing with Logstash
-
D1-Advanced Log Collection and Shipping-Logstash Task : Centralized Log Collection & Indexing with Logstash
-
1๏ธโฃ Windows Logs Collection :
-
Install Winlogbeat on a Windows machine.
-
Set up Winlogbeat to send Windows event logs to Logstash server.
-
-
2๏ธโฃ Linux Logs Collection :
-
Install Filebeat on Linux machine.
-
Preparing or using Nginx dummy logs.
-
Set up Filebeat to send Nginx logs to Logstash server.
-
-
3๏ธโฃ Logstash to Elasticsearch :
-
Set up Logstash to receive logs from both Winlogbeat and Filebeat. Send logs from Logstash to Elasticsearch making sure:
-
Each log source is stored in a separate index.
-
Example:
-
Windows logs โ
windows-logs -
Nginx logs โ
nginx-logs
-
-
Lab 2 : Windows Event Forwarding & ELK Integration
- Windows Logs Pipeline: WEF โ Logstash โ Elasticsearch
| Device | Role | Executed Tasks |
|---|---|---|
| AD Server (Windows Server) | Log Source | - Generating Windows Security/Application logs. - Enabling Audit Policy. - Activating WinRM and configuring GPO to point to the Log Collector. |
| Log Collector (Windows) | WEF Collector + Logstash | - Receiving WEF logs from the AD via Forwarded Events. - Installing Logstash to process these logs. - Sending data to the ELK Server (Linux). |
| ELK Server (Linux) | Storage and Analysis | - Receiving logs from Logstash. - Storing and analyzing data using Elasticsearch/Kibana. |
Lab 3 : Linux Log Collection & Analysis with Fluent Bit and Elasticsearch
-
Send linux auth logs via syslog to fluentbit syslog listener and parse all of them
-
Configure auditd on linux and send admin activities logs to fluentbit Parsethem .
-
Then outptut parsed auth logs and admin activites parsed logs to elasticsearch using fluentbit .