k3s-gitops/apps/demo-nginx/docs/README.md

🚀 Demo Nginx - CI/CD Pipeline Documentation

📋 Overview

This is a complete CI/CD pipeline for deploying Nginx application using:

• Jenkins - CI/CD orchestration
• Gitea - Git repository and GitOps manifests
• ArgoCD - GitOps deployment
• Kubernetes - Container orchestration
• Docker - Container runtime

---

🏗️ Architecture

┌─────────────┐      ┌──────────────┐      ┌─────────────┐      ┌────────────┐
│   Developer │─────▶│    Gitea     │─────▶│   Jenkins   │─────▶│   Docker   │
│   (commit)  │      │ (k3s-gitops) │      │  (pipeline) │      │  Registry  │
└─────────────┘      └──────────────┘      └──────┬──────┘      └─────┬──────┘
                                                   │                    │
                                                   │ Update             │ Pull
                                                   │ manifests          │ image
                                                   ▼                    │
                                            ┌──────────────┐            │
                                            │    Gitea     │            │
                                            │  (manifests) │            │
                                            └──────┬───────┘            │
                                                   │                    │
                                                   │ Watch              │
                                                   ▼                    │
                                            ┌──────────────┐            │
                                            │   ArgoCD     │────────────┘
                                            │   (sync)     │
                                            └──────┬───────┘
                                                   │
                                                   │ Apply
                                                   ▼
                                            ┌──────────────┐
                                            │  Kubernetes  │
                                            │   Cluster    │
                                            └──────────────┘

---

📁 Repository Structure

k3s-gitops/
└── apps/
    └── demo-nginx/
        ├── namespace.yaml        # Kubernetes namespace
        ├── deployment.yaml       # Deployment + Service
        ├── ingress.yaml         # Ingress with TLS
        ├── application.yaml     # ArgoCD Application
        ├── Jenkinsfile          # CI/CD Pipeline
        └── README.md            # This file

---

🔧 Prerequisites

1. Jenkins Configuration

Required Plugins:

• Git Plugin
• Docker Plugin
• Kubernetes Plugin
• Pipeline Plugin
• Credentials Plugin

Required Credentials in Jenkins:

Docker Registry Credentials:

ID: docker-registry-credentials
Type: Username with password
Username: <your-docker-hub-username>
Password: <your-docker-hub-token>

Gitea Credentials:

ID: gitea-credentials
Type: Username with password
Username: admin
Password: <gitea-password>

Create Credentials in Jenkins:

1. Jenkins → Manage Jenkins → Credentials
2. (global) → Add Credentials
3. Select "Username with password"
4. Fill in details and save

2. Docker Registry

Option A: Docker Hub

• Free for public images
• URL: docker.io
• Create account: https://hub.docker.com/signup

Option B: Harbor (Local)

• Already installed in cluster
• URL: harbor.thedevops.dev
• Create project: demo-nginx

3. Kubernetes Access

Jenkins needs kubectl access to cluster:

# Copy kubeconfig to Jenkins
kubectl create secret generic kubeconfig \
  --from-file=config=$HOME/.kube/config \
  -n jenkins

# Or configure ServiceAccount with proper RBAC

---

🚀 Setup Instructions

Step 1: Apply ArgoCD Application

kubectl apply -f apps/demo-nginx/application.yaml

Verify:

kubectl get application demo-nginx -n argocd

Step 2: Configure Jenkins Job

Method A: Multibranch Pipeline (Recommended)

1. Jenkins → New Item

2. Enter name: demo-nginx

3. Select: Multibranch Pipeline

4. Click: OK

5. Branch Sources:

   • Add source: Git
   • Project Repository: http://gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops
   • Credentials: Select gitea-credentials
   • Behaviors: Discover branches

6. Build Configuration:

   • Mode: by Jenkinsfile
   • Script Path: apps/demo-nginx/Jenkinsfile

7. Scan Multibranch Pipeline Triggers:

   • ✅ Periodically if not otherwise run
   • Interval: 1 minute

8. Save

Method B: Pipeline Job

1. Jenkins → New Item

2. Enter name: demo-nginx-pipeline

3. Select: Pipeline

4. Click: OK

5. Pipeline:

   • Definition: Pipeline script from SCM
   • SCM: Git
   • Repository URL: http://gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops
   • Credentials: gitea-credentials
   • Branch: */main
   • Script Path: apps/demo-nginx/Jenkinsfile

6. Build Triggers:

   • ✅ Poll SCM
   • Schedule: H/5 * * * * (every 5 minutes)

7. Save

Step 3: Configure Webhook (Optional)

For instant builds on git push:

In Gitea:

1. Repository Settings → Webhooks
2. Add Webhook → Gitea
3. URL: http://jenkins.jenkins.svc.cluster.local:8080/gitea-webhook/post
4. Events: Push
5. Active: ✅

In Jenkins:

• Build Triggers → ✅ Build when a change is pushed to Gitea

---

🏃 Running the Pipeline

Trigger Build

Option 1: Manual Trigger

# In Jenkins UI
Jenkins → demo-nginx → Build Now

Option 2: Git Push

# Make any change to trigger pipeline
git commit --allow-empty -m "Trigger pipeline"
git push origin main

Option 3: API Trigger

curl -X POST http://jenkins.jenkins.svc.cluster.local:8080/job/demo-nginx/build \
  --user admin:your-token

---

📊 Pipeline Stages

1. Checkout Source

• Clones repository
• Creates Dockerfile and nginx.conf

2. Build Docker Image

• Builds image with version tag
• Tags: ${IMAGE_TAG} and latest

3. Test Image

• Runs container locally
• Tests HTTP endpoints
• Verifies health check

4. Push to Registry

• Only on main branch
• Logs in to Docker registry
• Pushes both tags
• Cleans up credentials

5. Update GitOps Manifests

• Only on main branch
• Clones k3s-gitops repository
• Updates deployment.yaml with new image tag
• Commits and pushes changes

6. Wait for ArgoCD Sync

• Monitors ArgoCD application status
• Waits for Synced + Healthy state
• Timeout: 5 minutes

7. Verify Deployment

• Checks rollout status
• Verifies deployed image version
• Lists running pods

---

🔍 Pipeline Variables

You can customize these in Jenkinsfile:

Variable	Default	Description
APP_NAME	demo-nginx	Application name
NAMESPACE	demo-app	Kubernetes namespace
DOCKER_REGISTRY	docker.io	Docker registry URL
DOCKER_REPO	vladimiras	Docker repository/username
GITEA_URL	http://gitea-http...	Gitea service URL
GITEA_REPO	admin/k3s-gitops	GitOps repository

To customize:

environment {
    DOCKER_REGISTRY = 'harbor.thedevops.dev'
    DOCKER_REPO = 'library'
}

---

🧪 Testing

Test Locally

# Build image
docker build -t demo-nginx:test .

# Run container
docker run -d -p 8080:80 demo-nginx:test

# Test
curl http://localhost:8080/
curl http://localhost:8080/health

# Cleanup
docker stop $(docker ps -q --filter ancestor=demo-nginx:test)

Test in Cluster

# Port-forward to service
kubectl port-forward svc/demo-nginx 8080:80 -n demo-app

# Test
curl http://localhost:8080/

Access via Ingress

# Add to /etc/hosts
echo "5.182.17.194 demo-nginx.thedevops.dev" | sudo tee -a /etc/hosts

# Test (after DNS is configured)
curl https://demo-nginx.thedevops.dev/

---

📈 Monitoring

Check Pipeline Status

# Jenkins CLI
java -jar jenkins-cli.jar -s http://jenkins:8080/ \
  -auth admin:token \
  build demo-nginx -s -v

# Or via Web UI
# Jenkins → demo-nginx → Build History

Check ArgoCD Sync

# Get application status
kubectl get application demo-nginx -n argocd -o yaml

# Watch sync status
watch -n 2 'kubectl get application demo-nginx -n argocd'

# ArgoCD UI
# Open: https://argocd.thedevops.dev
# Applications → demo-nginx

Check Deployment

# Deployment status
kubectl get deployment demo-nginx -n demo-app

# Pods
kubectl get pods -n demo-app -l app=demo-nginx

# Logs
kubectl logs -n demo-app -l app=demo-nginx --tail=50

# Events
kubectl get events -n demo-app --sort-by='.lastTimestamp'

---

🐛 Troubleshooting

Pipeline Fails at "Build Docker Image"

Problem: Docker not available in Jenkins

Solution:

# Ensure Docker socket is mounted in Jenkins pod
kubectl edit deployment jenkins -n jenkins

# Add volume:
volumes:
- name: docker-sock
  hostPath:
    path: /var/run/docker.sock

# Add volumeMount:
volumeMounts:
- name: docker-sock
  mountPath: /var/run/docker.sock

Pipeline Fails at "Push to Registry"

Problem: Invalid credentials

Solution:

1. Verify credentials in Jenkins
2. Test manually:

docker login docker.io -u username -p password

Pipeline Fails at "Update GitOps Manifests"

Problem: Git authentication failed

Solution:

1. Check Gitea credentials in Jenkins
2. Test Git access:

git clone http://username:password@gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops

ArgoCD Not Syncing

Problem: ArgoCD not watching repository

Solution:

# Refresh ArgoCD app
kubectl patch application demo-nginx -n argocd \
  --type merge -p '{"operation":{"initiatedBy":{"username":"admin"},"sync":{"revision":"HEAD"}}}'

# Or via ArgoCD UI
# Click "Refresh" button

Deployment Not Rolling Out

Problem: Image pull error

Solution:

# Check pod events
kubectl describe pod -n demo-app -l app=demo-nginx

# Check if image exists
docker pull vladimiras/demo-nginx:main-123

# Check ImagePullSecrets (if using private registry)
kubectl get deployment demo-nginx -n demo-app -o yaml | grep -A 5 imagePullSecrets

---

🔐 Security Best Practices

1. Use Private Registry

environment {
    DOCKER_REGISTRY = 'harbor.thedevops.dev'
    DOCKER_REPO = 'library'
}

2. Scan Images for Vulnerabilities

Add to pipeline:

stage('Security Scan') {
    steps {
        sh 'trivy image ${DOCKER_REGISTRY}/${DOCKER_REPO}/${APP_NAME}:${IMAGE_TAG}'
    }
}

3. Use ImagePullSecrets

kubectl create secret docker-registry regcred \
  --docker-server=harbor.thedevops.dev \
  --docker-username=admin \
  --docker-password=password \
  -n demo-app

# Add to deployment.yaml:
spec:
  imagePullSecrets:
  - name: regcred

4. Don't Hardcode Secrets

Use Jenkins Credentials:

withCredentials([string(credentialsId: 'api-token', variable: 'TOKEN')]) {
    sh 'curl -H "Authorization: Bearer $TOKEN" api.example.com'
}

---

📝 Customization

Change Application

Edit Dockerfile in Jenkinsfile:

stage('Checkout Source') {
    steps {
        // Clone your app repository
        git branch: 'main', 
            url: 'http://gitea-http.gitea.svc.cluster.local:3000/admin/your-app'
    }
}

Add Tests

stage('Unit Tests') {
    steps {
        sh 'npm test'
    }
}

stage('Integration Tests') {
    steps {
        sh 'npm run test:integration'
    }
}

Add Linting

stage('Lint') {
    steps {
        sh 'npm run lint'
        sh 'hadolint Dockerfile'
    }
}

Multi-Environment Deploy

stage('Deploy to Dev') {
    when { branch 'develop' }
    steps {
        sh 'sed -i "s|namespace: .*|namespace: dev|" apps/demo-nginx/deployment.yaml'
        // ... update and push
    }
}

stage('Deploy to Production') {
    when { branch 'main' }
    steps {
        input message: 'Deploy to production?'
        // ... deploy
    }
}

---

🎯 Next Steps

1. ✅ Configure DNS: Point demo-nginx.thedevops.dev to your cluster IP
2. ✅ Set up monitoring: Add Prometheus metrics
3. ✅ Add alerts: Configure alerting for failures
4. ✅ Implement blue-green: Add traffic splitting
5. ✅ Add rollback: Implement automatic rollback on failures

---

📚 References

• Jenkins Pipeline Syntax
• ArgoCD Documentation
• Kubernetes Documentation
• Docker Documentation

---

🤝 Contributing

To improve this pipeline:

1. Fork the repository
2. Create feature branch
3. Make changes
4. Test thoroughly
5. Submit pull request

---

Happy Deploying! 🚀