Add apps/cluster-health-dashboard/readme.md
This commit is contained in:
634
apps/cluster-health-dashboard/readme.md
Normal file
634
apps/cluster-health-dashboard/readme.md
Normal file
@@ -0,0 +1,634 @@
|
|||||||
|
# 📊 Cluster Health Dashboard - Setup Guide
|
||||||
|
|
||||||
|
Complete setup guide for the Kubernetes Cluster Health Dashboard Jenkins pipeline.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 What This Dashboard Does
|
||||||
|
|
||||||
|
### Collects:
|
||||||
|
- ✅ Cluster information (version, nodes, namespaces, pods)
|
||||||
|
- ✅ Resource metrics from Prometheus (CPU, Memory, Network)
|
||||||
|
- ✅ Pod status across all namespaces
|
||||||
|
- ✅ Node capacity and usage
|
||||||
|
- ✅ Cost estimation (monthly)
|
||||||
|
- ✅ Health checks and issues detection
|
||||||
|
|
||||||
|
### Generates:
|
||||||
|
- 📊 Interactive HTML dashboard
|
||||||
|
- 📄 JSON report with all metrics
|
||||||
|
- 📱 Telegram summary notification
|
||||||
|
- 📧 Optional email report
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Prerequisites
|
||||||
|
|
||||||
|
### Required:
|
||||||
|
- ✅ Jenkins with Kubernetes plugin
|
||||||
|
- ✅ kubectl configured with cluster access
|
||||||
|
- ✅ Prometheus running in cluster (for metrics)
|
||||||
|
- ✅ jq installed on Jenkins agent
|
||||||
|
- ✅ curl installed on Jenkins agent
|
||||||
|
|
||||||
|
### Optional:
|
||||||
|
- ⚙️ Telegram bot (for notifications)
|
||||||
|
- ⚙️ Email configured in Jenkins
|
||||||
|
- ⚙️ Grafana (referenced in dashboard)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Setup Steps
|
||||||
|
|
||||||
|
### Step 1: Install Required Tools on Jenkins Agent
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# SSH to your Jenkins agent or use Jenkins shell
|
||||||
|
|
||||||
|
# Install jq (JSON processor)
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y jq
|
||||||
|
|
||||||
|
# Verify installations
|
||||||
|
jq --version
|
||||||
|
kubectl version --client
|
||||||
|
curl --version
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Configure Prometheus Access
|
||||||
|
|
||||||
|
**Option A: If Prometheus is in your cluster (recommended)**
|
||||||
|
|
||||||
|
Check if Prometheus is accessible:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From Jenkins agent or any pod in cluster
|
||||||
|
kubectl get svc -n monitoring
|
||||||
|
|
||||||
|
# Should see something like:
|
||||||
|
# prometheus-server ClusterIP 10.43.xxx.xxx <none> 80/TCP
|
||||||
|
```
|
||||||
|
|
||||||
|
**Option B: If Prometheus is external**
|
||||||
|
|
||||||
|
Update Jenkinsfile environment variables:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
environment {
|
||||||
|
PROMETHEUS_URL = 'http://your-prometheus-url:9090'
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Test Prometheus access:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From Jenkins agent
|
||||||
|
curl "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=up"
|
||||||
|
|
||||||
|
# Should return JSON with metrics
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Set Up Telegram Notifications (Optional)
|
||||||
|
|
||||||
|
If you already have bot from previous setup, skip this!
|
||||||
|
|
||||||
|
**A. Create Bot (if not done)**
|
||||||
|
1. Open Telegram → @BotFather
|
||||||
|
2. `/newbot`
|
||||||
|
3. Get token: `1234567890:ABC...`
|
||||||
|
|
||||||
|
**B. Get Chat ID**
|
||||||
|
1. Telegram → @userinfobot
|
||||||
|
2. Get your ID: `904518516`
|
||||||
|
|
||||||
|
**C. Add to Jenkins Credentials**
|
||||||
|
|
||||||
|
Jenkins → Manage Jenkins → Manage Credentials → Add:
|
||||||
|
|
||||||
|
**Credential 1:**
|
||||||
|
```
|
||||||
|
Kind: Secret text
|
||||||
|
Secret: 8347227871:AAHmkc--2ky2yEK80EGyIfpItKzV9zhGZSI
|
||||||
|
ID: telegram-bot-token
|
||||||
|
Description: Telegram Bot Token
|
||||||
|
```
|
||||||
|
|
||||||
|
**Credential 2:**
|
||||||
|
```
|
||||||
|
Kind: Secret text
|
||||||
|
Secret: 904518516
|
||||||
|
ID: telegram-chat-id
|
||||||
|
Description: Telegram Chat ID
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Adjust Cost Estimates
|
||||||
|
|
||||||
|
Edit Jenkinsfile to match your actual cloud costs:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
environment {
|
||||||
|
// Adjust these to your actual pricing
|
||||||
|
CPU_PRICE_PER_HOUR = '0.04' // $0.04 per vCPU/hour
|
||||||
|
MEMORY_PRICE_PER_GB_HOUR = '0.005' // $0.005 per GB/hour
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common pricing reference:**
|
||||||
|
- AWS t3.medium: ~$0.0416/hour (2 vCPU, 4GB RAM)
|
||||||
|
- DigitalOcean: $0.06/hour per vCPU, $0.007/GB RAM
|
||||||
|
- Local/Bare metal: $0 (or electricity cost)
|
||||||
|
|
||||||
|
### Step 5: Create Jenkins Pipeline
|
||||||
|
|
||||||
|
**A. Create New Pipeline Job**
|
||||||
|
|
||||||
|
1. Jenkins → New Item
|
||||||
|
2. Name: `cluster-health-dashboard`
|
||||||
|
3. Type: Pipeline
|
||||||
|
4. OK
|
||||||
|
|
||||||
|
**B. Configure Pipeline**
|
||||||
|
|
||||||
|
1. **Description:**
|
||||||
|
```
|
||||||
|
Daily cluster health monitoring and reporting.
|
||||||
|
Generates dashboard with metrics, costs, and health checks.
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Build Triggers:**
|
||||||
|
- ☑️ Build periodically
|
||||||
|
- Schedule: `0 8 * * 1-5` (8 AM weekdays)
|
||||||
|
|
||||||
|
3. **Pipeline:**
|
||||||
|
- Definition: Pipeline script from SCM
|
||||||
|
- SCM: Git
|
||||||
|
- Repository URL: `http://gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops`
|
||||||
|
- Credentials: `gitea-credentials`
|
||||||
|
- Branch: `*/main`
|
||||||
|
- Script Path: `apps/cluster-health-dashboard/Jenkinsfile`
|
||||||
|
|
||||||
|
**C. Or use Pipeline Script Directly**
|
||||||
|
|
||||||
|
If you want to test first without Git:
|
||||||
|
1. Definition: Pipeline script
|
||||||
|
2. Copy entire Jenkinsfile content into the script box
|
||||||
|
3. Save
|
||||||
|
|
||||||
|
### Step 6: Add to GitOps Repository
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# On your local machine
|
||||||
|
cd ~/projects/k3s-gitops
|
||||||
|
|
||||||
|
# Create directory
|
||||||
|
mkdir -p apps/cluster-health-dashboard
|
||||||
|
|
||||||
|
# Copy Jenkinsfile
|
||||||
|
cp /path/to/Jenkinsfile apps/cluster-health-dashboard/
|
||||||
|
|
||||||
|
# Commit
|
||||||
|
git add apps/cluster-health-dashboard/
|
||||||
|
git commit -m "feat: add cluster health dashboard pipeline"
|
||||||
|
git push origin main
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧪 Testing
|
||||||
|
|
||||||
|
### Test 1: Manual Run (First Time)
|
||||||
|
|
||||||
|
1. Jenkins → cluster-health-dashboard → Build with Parameters
|
||||||
|
2. Set:
|
||||||
|
- REPORT_PERIOD: `24h`
|
||||||
|
- SEND_EMAIL: `false` (for first test)
|
||||||
|
- SEND_TELEGRAM: `true`
|
||||||
|
3. Build Now
|
||||||
|
|
||||||
|
**Watch Console Output:**
|
||||||
|
```
|
||||||
|
🚀 Starting Cluster Health Dashboard generation...
|
||||||
|
📋 Collecting cluster information...
|
||||||
|
Cluster version: v1.28.0
|
||||||
|
Nodes: 3
|
||||||
|
Namespaces: 14
|
||||||
|
Pods: 67
|
||||||
|
📈 Querying Prometheus for metrics...
|
||||||
|
✅ Dashboard generated
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test 2: Check Generated Dashboard
|
||||||
|
|
||||||
|
After build completes:
|
||||||
|
|
||||||
|
1. Jenkins → cluster-health-dashboard → Build #1
|
||||||
|
2. Click "Cluster Health Dashboard" (left sidebar)
|
||||||
|
3. Should see beautiful HTML dashboard! 🎨
|
||||||
|
|
||||||
|
### Test 3: Check Telegram Notification
|
||||||
|
|
||||||
|
You should receive:
|
||||||
|
```
|
||||||
|
📊 Cluster Health Report
|
||||||
|
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
📋 Cluster Info
|
||||||
|
Version: v1.28.0
|
||||||
|
Nodes: 3
|
||||||
|
Namespaces: 14
|
||||||
|
Total Pods: 67
|
||||||
|
|
||||||
|
━━━━━━━━━━━━━━━━━━━━━━
|
||||||
|
💻 Resources
|
||||||
|
CPU Cores: 12
|
||||||
|
Memory: 48 GB
|
||||||
|
Avg CPU Usage: 23.5%
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test 4: Check Artifacts
|
||||||
|
|
||||||
|
1. Build #1 → Artifacts
|
||||||
|
2. Should see:
|
||||||
|
- `dashboard.html`
|
||||||
|
- `report.json`
|
||||||
|
- `namespace-stats.json`
|
||||||
|
- `all-pods.json`
|
||||||
|
- `node-resources.json`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Troubleshooting
|
||||||
|
|
||||||
|
### Issue 1: "Failed to query Prometheus"
|
||||||
|
|
||||||
|
**Symptoms:**
|
||||||
|
```
|
||||||
|
⚠️ Failed to query Prometheus: Connection refused
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check if Prometheus is running
|
||||||
|
kubectl get pods -n monitoring
|
||||||
|
|
||||||
|
# Check service
|
||||||
|
kubectl get svc -n monitoring
|
||||||
|
|
||||||
|
# Test connection from Jenkins pod
|
||||||
|
kubectl exec -it jenkins-0 -n jenkins -- \
|
||||||
|
curl "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=up"
|
||||||
|
```
|
||||||
|
|
||||||
|
**If Prometheus is in different namespace:**
|
||||||
|
|
||||||
|
Update Jenkinsfile:
|
||||||
|
```groovy
|
||||||
|
PROMETHEUS_URL = 'http://prometheus-server.YOUR_NAMESPACE.svc.cluster.local'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue 2: "jq: command not found"
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install jq on Jenkins agent
|
||||||
|
kubectl exec -it jenkins-0 -n jenkins -- apt-get update
|
||||||
|
kubectl exec -it jenkins-0 -n jenkins -- apt-get install -y jq
|
||||||
|
|
||||||
|
# Or add to Jenkins Dockerfile:
|
||||||
|
# RUN apt-get update && apt-get install -y jq
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue 3: "kubectl: command not found"
|
||||||
|
|
||||||
|
**Fix:**
|
||||||
|
|
||||||
|
Jenkins needs kubectl. Check installation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl exec -it jenkins-0 -n jenkins -- kubectl version --client
|
||||||
|
|
||||||
|
# If not installed, add to Jenkins image or install:
|
||||||
|
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
|
||||||
|
chmod +x kubectl
|
||||||
|
mv kubectl /usr/local/bin/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue 4: Dashboard shows "0" for all metrics
|
||||||
|
|
||||||
|
**Possible causes:**
|
||||||
|
1. Prometheus not accessible
|
||||||
|
2. Wrong Prometheus URL
|
||||||
|
3. No metrics in Prometheus
|
||||||
|
|
||||||
|
**Debug:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Test Prometheus query manually
|
||||||
|
curl "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=up"
|
||||||
|
|
||||||
|
# Check if metrics exist
|
||||||
|
curl "http://prometheus-server.monitoring.svc.cluster.local/api/v1/query?query=container_cpu_usage_seconds_total"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue 5: HTML Dashboard not showing
|
||||||
|
|
||||||
|
**Check:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Verify HTML Plugin is installed
|
||||||
|
Jenkins → Manage Jenkins → Manage Plugins → Installed
|
||||||
|
|
||||||
|
# Look for: HTML Publisher Plugin
|
||||||
|
|
||||||
|
# If not installed:
|
||||||
|
# Manage Plugins → Available → Search "HTML Publisher" → Install
|
||||||
|
```
|
||||||
|
|
||||||
|
### Issue 6: Telegram notifications not sending
|
||||||
|
|
||||||
|
**Check credentials:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Verify credentials exist
|
||||||
|
Jenkins → Manage Jenkins → Manage Credentials
|
||||||
|
|
||||||
|
# Should see:
|
||||||
|
# - telegram-bot-token
|
||||||
|
# - telegram-chat-id
|
||||||
|
|
||||||
|
# Test manually:
|
||||||
|
BOT_TOKEN="8347227871:AAHmkc--2ky2yEK80EGyIfpItKzV9zhGZSI"
|
||||||
|
CHAT_ID="904518516"
|
||||||
|
|
||||||
|
curl -X POST "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
|
||||||
|
-d chat_id="${CHAT_ID}" \
|
||||||
|
-d text="Test from terminal"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 Understanding the Dashboard
|
||||||
|
|
||||||
|
### Metrics Explained:
|
||||||
|
|
||||||
|
**Cluster Information:**
|
||||||
|
- **Kubernetes Version:** Your K8s version
|
||||||
|
- **Nodes:** Number of worker nodes
|
||||||
|
- **Namespaces:** Total namespaces
|
||||||
|
- **Total Pods:** All pods across cluster
|
||||||
|
|
||||||
|
**Resource Capacity:**
|
||||||
|
- **Total CPU Cores:** Sum of all node CPUs
|
||||||
|
- **Total Memory:** Sum of all node RAM
|
||||||
|
- **Avg CPU Usage:** Average CPU across containers
|
||||||
|
- **Progress Bar:** Visual CPU usage
|
||||||
|
|
||||||
|
**Pod Status:**
|
||||||
|
- **Running:** Healthy pods ✅
|
||||||
|
- **Pending:** Pods waiting to start ⏳
|
||||||
|
- **Failed:** Crashed pods ❌
|
||||||
|
- **Total Restarts:** Container restarts (high = problem)
|
||||||
|
|
||||||
|
**Monthly Costs:**
|
||||||
|
- Based on CPU cores and Memory GB
|
||||||
|
- Calculated using rates you configured
|
||||||
|
- Estimates infrastructure cost
|
||||||
|
|
||||||
|
**Health Checks:**
|
||||||
|
- High restart count (>10)
|
||||||
|
- Failed pods (>0)
|
||||||
|
- Pending pods (>5)
|
||||||
|
- High CPU usage (>80%)
|
||||||
|
|
||||||
|
**Resources by Namespace:**
|
||||||
|
- Table showing pod/container count per namespace
|
||||||
|
- Sorted by pod count (highest first)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎨 Customization
|
||||||
|
|
||||||
|
### Change Schedule
|
||||||
|
|
||||||
|
Edit cron trigger in Jenkinsfile:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
triggers {
|
||||||
|
cron('0 8 * * 1-5') // Weekdays 8 AM
|
||||||
|
|
||||||
|
// Examples:
|
||||||
|
// cron('0 */6 * * *') // Every 6 hours
|
||||||
|
// cron('0 9 * * MON') // Mondays 9 AM
|
||||||
|
// cron('0 0 * * *') // Daily midnight
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Add More Metrics
|
||||||
|
|
||||||
|
Add to Prometheus queries section:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
// Disk I/O
|
||||||
|
env.DISK_READ_MB = queryPrometheus(
|
||||||
|
"sum(rate(container_fs_reads_bytes_total[5m])) / 1024 / 1024"
|
||||||
|
)
|
||||||
|
|
||||||
|
// HTTP Requests (if you have metrics)
|
||||||
|
env.HTTP_REQUESTS_PER_SEC = queryPrometheus(
|
||||||
|
"sum(rate(http_requests_total[5m]))"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Then add to HTML dashboard:
|
||||||
|
|
||||||
|
```html
|
||||||
|
<div class="metric">
|
||||||
|
<span class="metric-label">Disk Read</span>
|
||||||
|
<span class="metric-value">${env.DISK_READ_MB} MB/s</span>
|
||||||
|
</div>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Change Colors/Styling
|
||||||
|
|
||||||
|
Edit CSS in `generateDashboardHTML()`:
|
||||||
|
|
||||||
|
```css
|
||||||
|
/* Change main gradient */
|
||||||
|
background: linear-gradient(135deg, #YOUR_COLOR1 0%, #YOUR_COLOR2 100%);
|
||||||
|
|
||||||
|
/* Change card colors */
|
||||||
|
.card h2 {
|
||||||
|
color: #YOUR_COLOR;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Add Email Recipients
|
||||||
|
|
||||||
|
Add to Jenkinsfile:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
post {
|
||||||
|
success {
|
||||||
|
emailext (
|
||||||
|
to: 'devops-team@company.com',
|
||||||
|
subject: "Cluster Health Report - ${new Date().format('yyyy-MM-dd')}",
|
||||||
|
body: '''
|
||||||
|
<h2>Daily Cluster Health Report</h2>
|
||||||
|
<p>Please see attached dashboard.</p>
|
||||||
|
''',
|
||||||
|
mimeType: 'text/html',
|
||||||
|
attachmentsPattern: '**/dashboard.html'
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📈 Usage Examples
|
||||||
|
|
||||||
|
### Weekly Review
|
||||||
|
|
||||||
|
```
|
||||||
|
Monday 8 AM → Dashboard generated
|
||||||
|
Review:
|
||||||
|
- Are costs increasing? Why?
|
||||||
|
- Any failed pods? Investigate
|
||||||
|
- CPU usage trending up? Scale?
|
||||||
|
- Restarts increasing? Bug in app?
|
||||||
|
```
|
||||||
|
|
||||||
|
### Cost Tracking
|
||||||
|
|
||||||
|
```
|
||||||
|
Week 1: $150/month
|
||||||
|
Week 2: $180/month ⚠️ (+20%)
|
||||||
|
→ Check namespace-stats.json
|
||||||
|
→ Which namespace grew?
|
||||||
|
→ Review pod counts
|
||||||
|
```
|
||||||
|
|
||||||
|
### Capacity Planning
|
||||||
|
|
||||||
|
```
|
||||||
|
Current: 12 CPU cores, 23.5% usage
|
||||||
|
If usage > 70% for 7 days:
|
||||||
|
→ Time to add nodes
|
||||||
|
→ Dashboard shows trend
|
||||||
|
```
|
||||||
|
|
||||||
|
### Health Monitoring
|
||||||
|
|
||||||
|
```
|
||||||
|
Dashboard shows:
|
||||||
|
❌ 5 pods in Failed state
|
||||||
|
⚠️ 15 container restarts
|
||||||
|
|
||||||
|
→ Click artifact → all-pods.json
|
||||||
|
→ Find which pods
|
||||||
|
→ kubectl logs <pod>
|
||||||
|
→ Fix issue
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔗 Integration with Other Tools
|
||||||
|
|
||||||
|
### Export to Grafana
|
||||||
|
|
||||||
|
Use `report.json`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Download report.json from Jenkins artifact
|
||||||
|
# Import to Grafana via JSON API datasource
|
||||||
|
# Create time-series dashboard
|
||||||
|
```
|
||||||
|
|
||||||
|
### Send to Slack
|
||||||
|
|
||||||
|
Add Slack webhook:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
post {
|
||||||
|
success {
|
||||||
|
sh """
|
||||||
|
curl -X POST ${SLACK_WEBHOOK_URL} \
|
||||||
|
-H 'Content-Type: application/json' \
|
||||||
|
-d '{
|
||||||
|
"text": "Daily Cluster Report: ${env.MONTHLY_TOTAL_COST} USD/month",
|
||||||
|
"attachments": [{
|
||||||
|
"color": "good",
|
||||||
|
"fields": [
|
||||||
|
{"title": "Nodes", "value": "${env.NODE_COUNT}", "short": true},
|
||||||
|
{"title": "Pods", "value": "${env.POD_COUNT}", "short": true}
|
||||||
|
]
|
||||||
|
}]
|
||||||
|
}'
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Store in Database
|
||||||
|
|
||||||
|
Parse JSON and insert:
|
||||||
|
|
||||||
|
```groovy
|
||||||
|
stage('Store in Database') {
|
||||||
|
steps {
|
||||||
|
script {
|
||||||
|
def report = readJSON file: "${OUTPUT_DIR}/report.json"
|
||||||
|
|
||||||
|
sh """
|
||||||
|
psql -h postgres -U metrics -d cluster_metrics -c "
|
||||||
|
INSERT INTO daily_reports (date, cpu_usage, pod_count, cost)
|
||||||
|
VALUES ('${report.generated_at}', ${report.resources.avg_cpu_usage_percent},
|
||||||
|
${report.cluster.total_pods}, ${report.costs.monthly_total_usd})
|
||||||
|
"
|
||||||
|
"""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Verification Checklist
|
||||||
|
|
||||||
|
After setup, verify:
|
||||||
|
|
||||||
|
- [ ] Jenkins job created
|
||||||
|
- [ ] First build succeeds
|
||||||
|
- [ ] HTML dashboard accessible
|
||||||
|
- [ ] Metrics show real data (not zeros)
|
||||||
|
- [ ] Telegram notification received
|
||||||
|
- [ ] Costs calculated correctly
|
||||||
|
- [ ] JSON report generated
|
||||||
|
- [ ] Namespace table populated
|
||||||
|
- [ ] Health checks working
|
||||||
|
- [ ] Schedule triggers correctly
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📚 Next Steps
|
||||||
|
|
||||||
|
### Enhancements:
|
||||||
|
1. **Historical Tracking** - Store reports in Git or database
|
||||||
|
2. **Alerts** - Trigger alerts on threshold breaches
|
||||||
|
3. **Comparison** - Compare week-over-week trends
|
||||||
|
4. **Recommendations** - Auto-suggest optimizations
|
||||||
|
5. **Deep Dive** - Per-namespace detailed reports
|
||||||
|
|
||||||
|
### Related Pipelines:
|
||||||
|
- Security Scanning (scan images from this report)
|
||||||
|
- Cleanup Pipeline (remove resources shown as unused)
|
||||||
|
- Backup Pipeline (backup based on importance shown here)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**You're all set! 🎉**
|
||||||
|
|
||||||
|
Run your first build and enjoy your cluster health dashboard! 📊✨
|
||||||
Reference in New Issue
Block a user