← Back to all products

Log Management Toolkit

$29

ELK/EFK stack configurations, log rotation scripts, structured logging templates, and log analysis queries.

18 filesv1.0.0
ShellYAMLMarkdownJSONConfigDockerNginx

Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

Interactive Reader (Free Preview) ⚙ Try Demo Tools Download Free Sample

File Structure 18 files

log-management-toolkit/ ├── .env.example ├── LICENSE ├── README.md ├── docker-compose.yml ├── elasticsearch/ │ └── elasticsearch.yml ├── filebeat/ │ ├── filebeat.yml │ └── modules.d/ │ ├── nginx.yml │ └── system.yml ├── guides/ │ └── log-management-strategy.md ├── kibana/ │ └── export/ │ └── dashboards.ndjson ├── logstash/ │ ├── patterns/ │ │ └── custom-patterns │ └── pipeline/ │ ├── app-json.conf │ ├── main.conf │ └── nginx-access.conf └── scripts/ ├── backup-kibana.sh ├── rotate-indices.sh └── setup.sh

Documentation Preview README excerpt

Log Management Toolkit

Production-ready ELK stack deployment with structured logging pipelines and operational dashboards.

Complete Elasticsearch, Logstash, Kibana, and Filebeat configurations for centralized log management. Includes ready-to-use parsing pipelines for Nginx, application JSON logs, and system logs, plus operational scripts for index management, backups, and a comprehensive log strategy guide.

What You Get

  • Docker Compose stack — One-command ELK deployment with resource limits
  • 5 Logstash pipelines — Parse Nginx, JSON apps, syslog, and custom formats
  • Filebeat configuration — Ship logs from any server to your stack
  • Kibana dashboards — Pre-built visualizations for immediate insight
  • 3 operational scripts — Setup, index rotation, and Kibana backup
  • Strategy guide — Log levels, retention, alerting, and compliance

File Tree


log-management-toolkit/
├── README.md
├── LICENSE
├── manifest.json
├── .env.example
├── docker-compose.yml
├── elasticsearch/
│   └── elasticsearch.yml          # ES node configuration
├── logstash/
│   ├── pipeline/
│   │   ├── main.conf              # Main routing pipeline
│   │   ├── nginx-access.conf      # Nginx access log parser
│   │   └── app-json.conf          # JSON application log parser
│   └── patterns/
│       └── custom-patterns        # Custom grok patterns
├── filebeat/
│   ├── filebeat.yml               # Filebeat shipper config
│   └── modules.d/
│       ├── nginx.yml              # Nginx module config
│       └── system.yml             # System log module config
├── kibana/
│   └── export/
│       └── dashboards.ndjson      # Pre-built dashboards
├── scripts/
│   ├── setup.sh                   # Stack deployment script
│   ├── rotate-indices.sh          # Index lifecycle management
│   └── backup-kibana.sh           # Kibana saved objects backup
└── guides/
    └── log-management-strategy.md # Logging strategy & best practices

Getting Started

1. Configure environment


cp .env.example .env
# Edit .env with your settings (passwords, retention, memory limits)
vim .env

2. Deploy the stack

... continues with setup instructions, usage examples, and more.

Code Sample .yml preview

elasticsearch/elasticsearch.yml # ============================================================================= # elasticsearch.yml — Elasticsearch Node Configuration # Part of: Log Management Toolkit by Datanest Digital # License: MIT | https://datanest.dev # # This file is mounted into the ES container. # Sensitive values are set via environment variables in docker-compose.yml. # ============================================================================= # ---- Cluster ---- cluster.name: ${CLUSTER_NAME:log-management} node.name: ${NODE_NAME:es-node-01} # ---- Network ---- network.host: 0.0.0.0 http.port: 9200 transport.port: 9300 # ---- Discovery ---- # Single-node mode (set via docker-compose environment) # For multi-node, uncomment and configure: # discovery.seed_hosts: ["es-node-02", "es-node-03"] # cluster.initial_master_nodes: ["es-node-01", "es-node-02", "es-node-03"] # ---- Paths ---- path.data: /usr/share/elasticsearch/data path.logs: /usr/share/elasticsearch/logs # ---- Memory ---- # Lock memory to prevent swapping (critical for performance) bootstrap.memory_lock: true # ---- Index Defaults ---- # Default number of shards and replicas for new indices # Single-node: 1 shard, 0 replicas # Multi-node: adjust based on cluster size action.auto_create_index: true # ---- Thread Pools ---- # Increase for high-ingestion workloads # thread_pool.write.queue_size: 1000 # ---- Circuit Breakers ---- # Prevent OOM by limiting memory usage per request indices.breaker.total.use_real_memory: true # ---- Logging ---- # Reduce noisy deprecation warnings in production logger.org.elasticsearch.deprecation.search.slow: WARN # ... 8 more lines ...