594 lines
13 KiB
Markdown
594 lines
13 KiB
Markdown
# 🚀 Demo Nginx - CI/CD Pipeline Documentation
|
|
|
|
## 📋 Overview
|
|
|
|
This is a complete CI/CD pipeline for deploying Nginx application using:
|
|
- **Jenkins** - CI/CD orchestration
|
|
- **Gitea** - Git repository and GitOps manifests
|
|
- **ArgoCD** - GitOps deployment
|
|
- **Kubernetes** - Container orchestration
|
|
- **Docker** - Container runtime
|
|
|
|
---
|
|
|
|
## 🏗️ Architecture
|
|
|
|
```
|
|
┌─────────────┐ ┌──────────────┐ ┌─────────────┐ ┌────────────┐
|
|
│ Developer │─────▶│ Gitea │─────▶│ Jenkins │─────▶│ Docker │
|
|
│ (commit) │ │ (k3s-gitops) │ │ (pipeline) │ │ Registry │
|
|
└─────────────┘ └──────────────┘ └──────┬──────┘ └─────┬──────┘
|
|
│ │
|
|
│ Update │ Pull
|
|
│ manifests │ image
|
|
▼ │
|
|
┌──────────────┐ │
|
|
│ Gitea │ │
|
|
│ (manifests) │ │
|
|
└──────┬───────┘ │
|
|
│ │
|
|
│ Watch │
|
|
▼ │
|
|
┌──────────────┐ │
|
|
│ ArgoCD │────────────┘
|
|
│ (sync) │
|
|
└──────┬───────┘
|
|
│
|
|
│ Apply
|
|
▼
|
|
┌──────────────┐
|
|
│ Kubernetes │
|
|
│ Cluster │
|
|
└──────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 📁 Repository Structure
|
|
|
|
```
|
|
k3s-gitops/
|
|
└── apps/
|
|
└── demo-nginx/
|
|
├── namespace.yaml # Kubernetes namespace
|
|
├── deployment.yaml # Deployment + Service
|
|
├── ingress.yaml # Ingress with TLS
|
|
├── application.yaml # ArgoCD Application
|
|
├── Jenkinsfile # CI/CD Pipeline
|
|
└── README.md # This file
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 Prerequisites
|
|
|
|
### 1. Jenkins Configuration
|
|
|
|
#### Required Plugins:
|
|
- Git Plugin
|
|
- Docker Plugin
|
|
- Kubernetes Plugin
|
|
- Pipeline Plugin
|
|
- Credentials Plugin
|
|
|
|
#### Required Credentials in Jenkins:
|
|
|
|
**Docker Registry Credentials:**
|
|
```
|
|
ID: docker-registry-credentials
|
|
Type: Username with password
|
|
Username: <your-docker-hub-username>
|
|
Password: <your-docker-hub-token>
|
|
```
|
|
|
|
**Gitea Credentials:**
|
|
```
|
|
ID: gitea-credentials
|
|
Type: Username with password
|
|
Username: admin
|
|
Password: <gitea-password>
|
|
```
|
|
|
|
#### Create Credentials in Jenkins:
|
|
1. Jenkins → Manage Jenkins → Credentials
|
|
2. (global) → Add Credentials
|
|
3. Select "Username with password"
|
|
4. Fill in details and save
|
|
|
|
### 2. Docker Registry
|
|
|
|
**Option A: Docker Hub**
|
|
- Free for public images
|
|
- URL: docker.io
|
|
- Create account: https://hub.docker.com/signup
|
|
|
|
**Option B: Harbor (Local)**
|
|
- Already installed in cluster
|
|
- URL: harbor.thedevops.dev
|
|
- Create project: demo-nginx
|
|
|
|
### 3. Kubernetes Access
|
|
|
|
Jenkins needs kubectl access to cluster:
|
|
|
|
```bash
|
|
# Copy kubeconfig to Jenkins
|
|
kubectl create secret generic kubeconfig \
|
|
--from-file=config=$HOME/.kube/config \
|
|
-n jenkins
|
|
|
|
# Or configure ServiceAccount with proper RBAC
|
|
```
|
|
|
|
---
|
|
|
|
## 🚀 Setup Instructions
|
|
|
|
### Step 1: Apply ArgoCD Application
|
|
|
|
```bash
|
|
kubectl apply -f apps/demo-nginx/application.yaml
|
|
```
|
|
|
|
Verify:
|
|
```bash
|
|
kubectl get application demo-nginx -n argocd
|
|
```
|
|
|
|
### Step 2: Configure Jenkins Job
|
|
|
|
#### Method A: Multibranch Pipeline (Recommended)
|
|
|
|
1. **Jenkins → New Item**
|
|
2. **Enter name**: `demo-nginx`
|
|
3. **Select**: Multibranch Pipeline
|
|
4. **Click**: OK
|
|
|
|
5. **Branch Sources**:
|
|
- Add source: Git
|
|
- Project Repository: `http://gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops`
|
|
- Credentials: Select gitea-credentials
|
|
- Behaviors: Discover branches
|
|
|
|
6. **Build Configuration**:
|
|
- Mode: by Jenkinsfile
|
|
- Script Path: `apps/demo-nginx/Jenkinsfile`
|
|
|
|
7. **Scan Multibranch Pipeline Triggers**:
|
|
- ✅ Periodically if not otherwise run
|
|
- Interval: 1 minute
|
|
|
|
8. **Save**
|
|
|
|
#### Method B: Pipeline Job
|
|
|
|
1. **Jenkins → New Item**
|
|
2. **Enter name**: `demo-nginx-pipeline`
|
|
3. **Select**: Pipeline
|
|
4. **Click**: OK
|
|
|
|
5. **Pipeline**:
|
|
- Definition: Pipeline script from SCM
|
|
- SCM: Git
|
|
- Repository URL: `http://gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops`
|
|
- Credentials: gitea-credentials
|
|
- Branch: `*/main`
|
|
- Script Path: `apps/demo-nginx/Jenkinsfile`
|
|
|
|
6. **Build Triggers**:
|
|
- ✅ Poll SCM
|
|
- Schedule: `H/5 * * * *` (every 5 minutes)
|
|
|
|
7. **Save**
|
|
|
|
### Step 3: Configure Webhook (Optional)
|
|
|
|
For instant builds on git push:
|
|
|
|
**In Gitea:**
|
|
1. Repository Settings → Webhooks
|
|
2. Add Webhook → Gitea
|
|
3. URL: `http://jenkins.jenkins.svc.cluster.local:8080/gitea-webhook/post`
|
|
4. Events: Push
|
|
5. Active: ✅
|
|
|
|
**In Jenkins:**
|
|
- Build Triggers → ✅ Build when a change is pushed to Gitea
|
|
|
|
---
|
|
|
|
## 🏃 Running the Pipeline
|
|
|
|
### Trigger Build
|
|
|
|
**Option 1: Manual Trigger**
|
|
```bash
|
|
# In Jenkins UI
|
|
Jenkins → demo-nginx → Build Now
|
|
```
|
|
|
|
**Option 2: Git Push**
|
|
```bash
|
|
# Make any change to trigger pipeline
|
|
git commit --allow-empty -m "Trigger pipeline"
|
|
git push origin main
|
|
```
|
|
|
|
**Option 3: API Trigger**
|
|
```bash
|
|
curl -X POST http://jenkins.jenkins.svc.cluster.local:8080/job/demo-nginx/build \
|
|
--user admin:your-token
|
|
```
|
|
|
|
---
|
|
|
|
## 📊 Pipeline Stages
|
|
|
|
### 1. Checkout Source
|
|
- Clones repository
|
|
- Creates Dockerfile and nginx.conf
|
|
|
|
### 2. Build Docker Image
|
|
- Builds image with version tag
|
|
- Tags: `${IMAGE_TAG}` and `latest`
|
|
|
|
### 3. Test Image
|
|
- Runs container locally
|
|
- Tests HTTP endpoints
|
|
- Verifies health check
|
|
|
|
### 4. Push to Registry
|
|
- **Only on main branch**
|
|
- Logs in to Docker registry
|
|
- Pushes both tags
|
|
- Cleans up credentials
|
|
|
|
### 5. Update GitOps Manifests
|
|
- **Only on main branch**
|
|
- Clones k3s-gitops repository
|
|
- Updates deployment.yaml with new image tag
|
|
- Commits and pushes changes
|
|
|
|
### 6. Wait for ArgoCD Sync
|
|
- Monitors ArgoCD application status
|
|
- Waits for Synced + Healthy state
|
|
- Timeout: 5 minutes
|
|
|
|
### 7. Verify Deployment
|
|
- Checks rollout status
|
|
- Verifies deployed image version
|
|
- Lists running pods
|
|
|
|
---
|
|
|
|
## 🔍 Pipeline Variables
|
|
|
|
You can customize these in Jenkinsfile:
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| APP_NAME | demo-nginx | Application name |
|
|
| NAMESPACE | demo-app | Kubernetes namespace |
|
|
| DOCKER_REGISTRY | docker.io | Docker registry URL |
|
|
| DOCKER_REPO | vladimiras | Docker repository/username |
|
|
| GITEA_URL | http://gitea-http... | Gitea service URL |
|
|
| GITEA_REPO | admin/k3s-gitops | GitOps repository |
|
|
|
|
**To customize:**
|
|
```groovy
|
|
environment {
|
|
DOCKER_REGISTRY = 'harbor.thedevops.dev'
|
|
DOCKER_REPO = 'library'
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 🧪 Testing
|
|
|
|
### Test Locally
|
|
|
|
```bash
|
|
# Build image
|
|
docker build -t demo-nginx:test .
|
|
|
|
# Run container
|
|
docker run -d -p 8080:80 demo-nginx:test
|
|
|
|
# Test
|
|
curl http://localhost:8080/
|
|
curl http://localhost:8080/health
|
|
|
|
# Cleanup
|
|
docker stop $(docker ps -q --filter ancestor=demo-nginx:test)
|
|
```
|
|
|
|
### Test in Cluster
|
|
|
|
```bash
|
|
# Port-forward to service
|
|
kubectl port-forward svc/demo-nginx 8080:80 -n demo-app
|
|
|
|
# Test
|
|
curl http://localhost:8080/
|
|
```
|
|
|
|
### Access via Ingress
|
|
|
|
```bash
|
|
# Add to /etc/hosts
|
|
echo "5.182.17.194 demo-nginx.thedevops.dev" | sudo tee -a /etc/hosts
|
|
|
|
# Test (after DNS is configured)
|
|
curl https://demo-nginx.thedevops.dev/
|
|
```
|
|
|
|
---
|
|
|
|
## 📈 Monitoring
|
|
|
|
### Check Pipeline Status
|
|
|
|
```bash
|
|
# Jenkins CLI
|
|
java -jar jenkins-cli.jar -s http://jenkins:8080/ \
|
|
-auth admin:token \
|
|
build demo-nginx -s -v
|
|
|
|
# Or via Web UI
|
|
# Jenkins → demo-nginx → Build History
|
|
```
|
|
|
|
### Check ArgoCD Sync
|
|
|
|
```bash
|
|
# Get application status
|
|
kubectl get application demo-nginx -n argocd -o yaml
|
|
|
|
# Watch sync status
|
|
watch -n 2 'kubectl get application demo-nginx -n argocd'
|
|
|
|
# ArgoCD UI
|
|
# Open: https://argocd.thedevops.dev
|
|
# Applications → demo-nginx
|
|
```
|
|
|
|
### Check Deployment
|
|
|
|
```bash
|
|
# Deployment status
|
|
kubectl get deployment demo-nginx -n demo-app
|
|
|
|
# Pods
|
|
kubectl get pods -n demo-app -l app=demo-nginx
|
|
|
|
# Logs
|
|
kubectl logs -n demo-app -l app=demo-nginx --tail=50
|
|
|
|
# Events
|
|
kubectl get events -n demo-app --sort-by='.lastTimestamp'
|
|
```
|
|
|
|
---
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### Pipeline Fails at "Build Docker Image"
|
|
|
|
**Problem:** Docker not available in Jenkins
|
|
|
|
**Solution:**
|
|
```bash
|
|
# Ensure Docker socket is mounted in Jenkins pod
|
|
kubectl edit deployment jenkins -n jenkins
|
|
|
|
# Add volume:
|
|
volumes:
|
|
- name: docker-sock
|
|
hostPath:
|
|
path: /var/run/docker.sock
|
|
|
|
# Add volumeMount:
|
|
volumeMounts:
|
|
- name: docker-sock
|
|
mountPath: /var/run/docker.sock
|
|
```
|
|
|
|
### Pipeline Fails at "Push to Registry"
|
|
|
|
**Problem:** Invalid credentials
|
|
|
|
**Solution:**
|
|
1. Verify credentials in Jenkins
|
|
2. Test manually:
|
|
```bash
|
|
docker login docker.io -u username -p password
|
|
```
|
|
|
|
### Pipeline Fails at "Update GitOps Manifests"
|
|
|
|
**Problem:** Git authentication failed
|
|
|
|
**Solution:**
|
|
1. Check Gitea credentials in Jenkins
|
|
2. Test Git access:
|
|
```bash
|
|
git clone http://username:password@gitea-http.gitea.svc.cluster.local:3000/admin/k3s-gitops
|
|
```
|
|
|
|
### ArgoCD Not Syncing
|
|
|
|
**Problem:** ArgoCD not watching repository
|
|
|
|
**Solution:**
|
|
```bash
|
|
# Refresh ArgoCD app
|
|
kubectl patch application demo-nginx -n argocd \
|
|
--type merge -p '{"operation":{"initiatedBy":{"username":"admin"},"sync":{"revision":"HEAD"}}}'
|
|
|
|
# Or via ArgoCD UI
|
|
# Click "Refresh" button
|
|
```
|
|
|
|
### Deployment Not Rolling Out
|
|
|
|
**Problem:** Image pull error
|
|
|
|
**Solution:**
|
|
```bash
|
|
# Check pod events
|
|
kubectl describe pod -n demo-app -l app=demo-nginx
|
|
|
|
# Check if image exists
|
|
docker pull vladimiras/demo-nginx:main-123
|
|
|
|
# Check ImagePullSecrets (if using private registry)
|
|
kubectl get deployment demo-nginx -n demo-app -o yaml | grep -A 5 imagePullSecrets
|
|
```
|
|
|
|
---
|
|
|
|
## 🔐 Security Best Practices
|
|
|
|
### 1. Use Private Registry
|
|
|
|
```groovy
|
|
environment {
|
|
DOCKER_REGISTRY = 'harbor.thedevops.dev'
|
|
DOCKER_REPO = 'library'
|
|
}
|
|
```
|
|
|
|
### 2. Scan Images for Vulnerabilities
|
|
|
|
Add to pipeline:
|
|
```groovy
|
|
stage('Security Scan') {
|
|
steps {
|
|
sh 'trivy image ${DOCKER_REGISTRY}/${DOCKER_REPO}/${APP_NAME}:${IMAGE_TAG}'
|
|
}
|
|
}
|
|
```
|
|
|
|
### 3. Use ImagePullSecrets
|
|
|
|
```bash
|
|
kubectl create secret docker-registry regcred \
|
|
--docker-server=harbor.thedevops.dev \
|
|
--docker-username=admin \
|
|
--docker-password=password \
|
|
-n demo-app
|
|
|
|
# Add to deployment.yaml:
|
|
spec:
|
|
imagePullSecrets:
|
|
- name: regcred
|
|
```
|
|
|
|
### 4. Don't Hardcode Secrets
|
|
|
|
Use Jenkins Credentials:
|
|
```groovy
|
|
withCredentials([string(credentialsId: 'api-token', variable: 'TOKEN')]) {
|
|
sh 'curl -H "Authorization: Bearer $TOKEN" api.example.com'
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 📝 Customization
|
|
|
|
### Change Application
|
|
|
|
Edit `Dockerfile` in Jenkinsfile:
|
|
```groovy
|
|
stage('Checkout Source') {
|
|
steps {
|
|
// Clone your app repository
|
|
git branch: 'main',
|
|
url: 'http://gitea-http.gitea.svc.cluster.local:3000/admin/your-app'
|
|
}
|
|
}
|
|
```
|
|
|
|
### Add Tests
|
|
|
|
```groovy
|
|
stage('Unit Tests') {
|
|
steps {
|
|
sh 'npm test'
|
|
}
|
|
}
|
|
|
|
stage('Integration Tests') {
|
|
steps {
|
|
sh 'npm run test:integration'
|
|
}
|
|
}
|
|
```
|
|
|
|
### Add Linting
|
|
|
|
```groovy
|
|
stage('Lint') {
|
|
steps {
|
|
sh 'npm run lint'
|
|
sh 'hadolint Dockerfile'
|
|
}
|
|
}
|
|
```
|
|
|
|
### Multi-Environment Deploy
|
|
|
|
```groovy
|
|
stage('Deploy to Dev') {
|
|
when { branch 'develop' }
|
|
steps {
|
|
sh 'sed -i "s|namespace: .*|namespace: dev|" apps/demo-nginx/deployment.yaml'
|
|
// ... update and push
|
|
}
|
|
}
|
|
|
|
stage('Deploy to Production') {
|
|
when { branch 'main' }
|
|
steps {
|
|
input message: 'Deploy to production?'
|
|
// ... deploy
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## 🎯 Next Steps
|
|
|
|
1. ✅ **Configure DNS**: Point `demo-nginx.thedevops.dev` to your cluster IP
|
|
2. ✅ **Set up monitoring**: Add Prometheus metrics
|
|
3. ✅ **Add alerts**: Configure alerting for failures
|
|
4. ✅ **Implement blue-green**: Add traffic splitting
|
|
5. ✅ **Add rollback**: Implement automatic rollback on failures
|
|
|
|
---
|
|
|
|
## 📚 References
|
|
|
|
- [Jenkins Pipeline Syntax](https://www.jenkins.io/doc/book/pipeline/syntax/)
|
|
- [ArgoCD Documentation](https://argo-cd.readthedocs.io/)
|
|
- [Kubernetes Documentation](https://kubernetes.io/docs/)
|
|
- [Docker Documentation](https://docs.docker.com/)
|
|
|
|
---
|
|
|
|
## 🤝 Contributing
|
|
|
|
To improve this pipeline:
|
|
1. Fork the repository
|
|
2. Create feature branch
|
|
3. Make changes
|
|
4. Test thoroughly
|
|
5. Submit pull request
|
|
|
|
---
|
|
|
|
**Happy Deploying! 🚀**
|