Ceds-HomeLab/ced-k3s-homelab
1
1
Fork 0
mirror of https://github.com/ced4568/ced-k3s-homelab.git synced 2026-08-13 06:24:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit: Ced's K3s HomeLab

Browse Source
This commit is contained in:
ced4568
2025-12-06 22:22:42 -06:00
commit 188651954e
20 changed files with 1078 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
k3s-homelab-full/Makefile
+37
View File
@@ -0,0 +1,37 @@
.PHONY: help label-nodes install-metallb install-ingress install-noc demo-app grafana-ingress prometheus-ingress all
help:
@echo "Ced's K3s HomeLab Makefile"
@echo
@echo "Targets:"
@echo " label-nodes - Label ingress/data/monitoring nodes"
@echo " install-metallb - Install and configure MetalLB"
@echo " install-ingress - Install ingress-nginx via Helm"
@echo " install-noc - Install Prometheus + Grafana (kube-prometheus-stack)"
@echo " demo-app - Deploy demo nginx app + ingress"
@echo " grafana-ingress - Apply Grafana ingress"
@echo " prometheus-ingress - Apply Prometheus ingress"
@echo " all - MetalLB + ingress + NOC + demo app + ingresses"
label-nodes:
./scripts/03_label_nodes.sh
install-metallb:
./scripts/10_install_metallb.sh
install-ingress:
./scripts/20_install_ingress_nginx.sh
install-noc:
./scripts/30_install_ceds_noc.sh
demo-app:
kubectl apply -f manifests/demo-app/demo-nginx.yaml
grafana-ingress:
kubectl apply -f manifests/ingress/grafana-ingress.yaml
prometheus-ingress:
kubectl apply -f manifests/ingress/prometheus-ingress.yaml
all: label-nodes install-metallb install-ingress install-noc demo-app grafana-ingress prometheus-ingress
k3s-homelab-full/README.md
+57
View File
@@ -0,0 +1,57 @@
# Ced's K3s HomeLab (Raspberry Pi Cluster)
This repo captures the working configuration of your HA K3s cluster:
- 3× control-plane nodes: `k3s-django-1/2/3`
- 9× worker nodes split into:
- Ingress: `k3s-node-1..3`
- Data: `k3s-node-4..6`
- Monitoring: `k3s-node-7..9`
- MetalLB for LoadBalancer IPs on your HomeLab VLAN
- ingress-nginx as the main ingress controller
- kube-prometheus-stack (Prometheus + Grafana + Alertmanager + exporters)
- Ced's NOC dashboards for Grafana
You can use this as a GitHub repo, import the dashboards into Grafana, and keep long‑term documentation of Ced's HomeLab.
## Quick Start (once K3s cluster is up)
On `k3s-django-1` with `KUBECONFIG` pointing at the cluster:
```bash
git clone <your-repo-or-copy-files>
cd k3s-homelab-full
# 1) Label nodes into groups
./scripts/03_label_nodes.sh
# 2) Install MetalLB
./scripts/10_install_metallb.sh
# 3) Install ingress-nginx
./scripts/20_install_ingress_nginx.sh
# 4) Install Ced's NOC (kube-prometheus-stack)
./scripts/30_install_ceds_noc.sh
# 5) Deploy demo nginx ingress + Grafana/Prometheus ingress rules
kubectl apply -f manifests/demo-app/demo-nginx.yaml
kubectl apply -f manifests/ingress/grafana-ingress.yaml
kubectl apply -f manifests/ingress/prometheus-ingress.yaml
```
Then update your desktop `hosts` file to point:
```text
10.10.30.251 demo.local grafana.local prometheus.local
```
## Structure
- `cluster-setup.md` – step-by-step narrative of the full setup.
- `kube-prom-values.yaml` – values file for `kube-prometheus-stack` (Ced's NOC).
- `dashboards/` – starter Grafana dashboards in JSON.
- `scripts/` – helper scripts to label nodes and install MetalLB, ingress, and NOC.
- `manifests/` – YAML manifests for MetalLB, demo app, and ingresses.
- `docs/per-node-notes.md` – inventory of each node (IP, role, hardware).
- `diagrams/ced-k3s-from-text.txt` – text that can be imported into draw.io ("Arrange → Insert → Advanced → From text").
k3s-homelab-full/bootstrap.sh
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
echo "=============================================="
echo " Ced's K3s HomeLab Bootstrap"
echo "=============================================="
echo
echo "This script assumes:"
echo " - K3s control-plane is already up (k3s-django-1/2/3)"
echo " - Worker nodes are joined (k3s-node-1..9)"
echo " - KUBECONFIG is set to point to the cluster"
echo
read -p "Press ENTER to continue or Ctrl+C to abort... " _
echo "[1/6] Labeling nodes into ingress/data/monitoring groups..."
./scripts/03_label_nodes.sh
echo "[2/6] Installing MetalLB (IP pool 10.10.30.251-254)..."
./scripts/10_install_metallb.sh
echo "[3/6] Installing ingress-nginx (LoadBalancer via MetalLB)..."
./scripts/20_install_ingress_nginx.sh
echo "[4/6] Installing Ced's NOC (kube-prometheus-stack)..."
./scripts/30_install_ceds_noc.sh
echo "[5/6] Deploying demo nginx app + ingress..."
kubectl apply -f manifests/demo-app/demo-nginx.yaml
echo "[6/6] Applying ingress for Grafana and Prometheus..."
kubectl apply -f manifests/ingress/grafana-ingress.yaml
kubectl apply -f manifests/ingress/prometheus-ingress.yaml
echo
echo "=============================================="
echo " Done!"
echo " - Demo app: http://demo.local"
echo " - Grafana: http://grafana.local"
echo " - Prometheus: http://prometheus.local"
echo
echo "Remember to add these to your desktop hosts file pointing to the"
echo "MetalLB ingress IP (e.g. 10.10.30.251):"
echo
echo " 10.10.30.251 demo.local grafana.local prometheus.local"
echo
echo "You can now log into Grafana and import dashboards from the"
echo "dashboards/ folder in this repo."
echo "=============================================="
k3s-homelab-full/cluster-setup.md
+340
View File
@@ -0,0 +1,340 @@
# Ced's K3s Cluster Setup – End-to-End
This document describes how your current K3s cluster is built so you can re‑create it later or explain it in GitHub/OneNote.
---
## 1. Base Networking
**HomeLab VLAN:** `10.10.30.0/24`
**Gateway:** UniFi Dream Router
**DHCP:** Configured to *exclude* `.251–.254` so MetalLB can safely use that range.
- Proxmox VE: `10.10.30.250`
- MetalLB pool: `10.10.30.251–10.10.30.254`
Subnets you have overall (for reference):
- Default: `10.10.1.0/24` (not used)
- Main: `10.10.10.0/24`
- IoT: `10.10.20.0/24`
- HomeLab: `10.10.30.0/24`
- Guest: `10.10.99.0/24`
---
## 2. K3s Control Plane
### Nodes
- `k3s-django-1` – `10.10.30.72` (control-plane, etcd, master)
- `k3s-django-2` – `10.10.30.245` (control-plane, etcd, master)
- `k3s-django-3` – `10.10.30.128` (control-plane, etcd, master)
All three are Debian 12 (Bookworm, ARM64 on Raspberry Pi).
### 2.1 Install K3s on the first control-plane node
On `k3s-django-1`:
```bash
curl -sfL https://get.k3s.io | sh -s - server
```
Then capture the version, token, and kubeconfig:
```bash
k3s --version
sudo cat /var/lib/rancher/k3s/server/node-token
mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config
```
Update the kubeconfig to point to the node's real IP instead of `127.0.0.1`:
```bash
sed -i 's/https:\/\/127.0.0.1:6443/https:\/\/10.10.30.72:6443/' ~/.kube/config
export KUBECONFIG=$HOME/.kube/config
```
Check:
```bash
kubectl get nodes -o wide
```
You should see `k3s-django-1` as `Ready control-plane,etcd,master`.
### 2.2 Join additional control-plane nodes
Get the join token from `k3s-django-1`:
```bash
TOKEN=$(sudo cat /var/lib/rancher/k3s/server/node-token)
echo "$TOKEN"
```
On `k3s-django-2` and `k3s-django-3`:
```bash
export K3S_URL="https://10.10.30.72:6443"
export K3S_TOKEN="<PASTE TOKEN FROM k3s-django-1>"
curl -sfL https://get.k3s.io | K3S_URL=$K3S_URL K3S_TOKEN=$K3S_TOKEN sh -s - server
```
Verify from `k3s-django-1`:
```bash
kubectl get nodes -o wide
```
You should now see all three `k3s-django-*` nodes as control-plane,etcd,master.
---
## 3. Worker Nodes
You have 9 worker nodes grouped as:
- Ingress: `k3s-node-1`, `k3s-node-2`, `k3s-node-3`
- Data: `k3s-node-4`, `k3s-node-5`, `k3s-node-6`
- Monitoring: `k3s-node-7`, `k3s-node-8`, `k3s-node-9`
### 3.1 Join workers as agents
On each worker node (`k3s-node-N`), use:
```bash
export K3S_URL="https://10.10.30.72:6443"
export K3S_TOKEN="<same token from k3s-django-1>"
curl -sfL https://get.k3s.io | K3S_URL=$K3S_URL K3S_TOKEN=$K3S_TOKEN sh -s - agent
```
Confirm from `k3s-django-1`:
```bash
kubectl get nodes -o wide
```
All 9 agents should show as `Ready` with no special roles yet.
### 3.2 Label nodes by function
From `k3s-django-1` (or any node with kubeconfig):
```bash
# Ingress group
kubectl label node k3s-node-1 ingress-node=true
kubectl label node k3s-node-2 ingress-node=true
kubectl label node k3s-node-3 ingress-node=true
# Data group
kubectl label node k3s-node-4 data-node=true
kubectl label node k3s-node-5 data-node=true
kubectl label node k3s-node-6 data-node=true
# Monitoring group
kubectl label node k3s-node-7 monitoring-node=true
kubectl label node k3s-node-8 monitoring-node=true
kubectl label node k3s-node-9 monitoring-node=true
kubectl get nodes -L ingress-node -L data-node -L monitoring-node
```
Additionally, you used `kubectl label node ...` to set the `ROLES` field (`ingress`, `data`, `monitoring`) for documentation.
---
## 4. MetalLB
### 4.1 Install MetalLB
```bash
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
```
Wait for pods:
```bash
kubectl get pods -n metallb-system -o wide
```
### 4.2 Configure IPAddressPool and L2Advertisement
MetalLB is configured with an address pool in `manifests/metallb/ipaddresspool.yaml`:
```yaml
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: ceds-pool
namespace: metallb-system
spec:
addresses:
- 10.10.30.251-10.10.30.254
```
and `manifests/metallb/l2advertisement.yaml`:
```yaml
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: ceds-l2
namespace: metallb-system
spec:
ipAddressPools:
- ceds-pool
```
Apply:
```bash
kubectl apply -f manifests/metallb/ipaddresspool.yaml
kubectl apply -f manifests/metallb/l2advertisement.yaml
```
Check:
```bash
kubectl get ipaddresspools.metallb.io -n metallb-system
kubectl get l2advertisements.metallb.io -n metallb-system
```
---
## 5. Ingress-NGINX
### 5.1 Install Helm & ingress-nginx
On `k3s-django-1`:
```bash
curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
```
Install ingress-nginx targeting the ingress node group:
```bash
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace --set controller.service.type=LoadBalancer --set controller.nodeSelector.ingress-node="true"
```
Check:
```bash
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginx ingress-nginx-controller
```
You should see `EXTERNAL-IP` assigned from MetalLB, e.g. `10.10.30.251`.
---
## 6. Demo Ingress App
`manifests/demo-app/demo-nginx.yaml` contains:
- A `Deployment` with 3 replicas of `nginx`
- A `ClusterIP` Service
- An Ingress pointing `demo.local` to the service via ingress-nginx
Apply:
```bash
kubectl apply -f manifests/demo-app/demo-nginx.yaml
```
Update your desktop hosts file:
```text
10.10.30.251 demo.local
```
Open `http://demo.local` in a browser and you should see the nginx welcome page, served via:
Desktop → MetalLB VIP (10.10.30.251) → ingress-nginx → demo-nginx Service → Pods.
---
## 7. Ced's NOC (kube-prometheus-stack)
Namespace:
```bash
kubectl create namespace monitoring
```
Helm repo:
```bash
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
```
Install with your custom values file:
```bash
helm install ceds-noc prometheus-community/kube-prometheus-stack --namespace monitoring -f kube-prom-values.yaml
```
`kube-prom-values.yaml` (included in this repo) pins Prometheus, Alertmanager, and Grafana to the `monitoring-node=true` nodes, configures PVCs for metrics and Grafana data, and exposes Grafana via a LoadBalancer (MetalLB).
Check pods & services:
```bash
kubectl get pods -n monitoring
kubectl get svc -n monitoring
```
You should see:
- `ceds-noc-grafana` as a `LoadBalancer` with an `EXTERNAL-IP` like `10.10.30.252`
- Prometheus, Alertmanager, kube-state-metrics, node exporters, etc.
Open Grafana at:
```text
http://10.10.30.252
```
Username: `admin`
Password: either the one in `kube-prom-values.yaml` (if set) or via:
```bash
kubectl --namespace monitoring get secret ceds-noc-grafana -o jsonpath="{.data.admin-password}" | base64 -d ; echo
```
---
## 8. Grafana Dashboards (Ced's NOC)
Use Grafana → Dashboards → New → Import, and upload:
- `dashboards/ced_cluster_overview.json`
- `dashboards/ced_nodes_detail.json`
- `dashboards/ced_ingress_metallb.json`
Choose the kube-prometheus-stack Prometheus datasource.
You now have a live NOC for:
- Cluster health (nodes, pods, API server)
- Per‑node CPU, RAM, disk, pod counts
- Ingress and MetalLB metrics
---
## 9. Future Enhancements
- Add Loki + Promtail for logs
- Add Prometheus alerting rules (node down, high CPU, low disk, etc)
- Add Alertmanager integrations (Discord/Slack/email)
- Use GitOps (FluxCD or ArgoCD) to manage this repo as the source of truth.
k3s-homelab-full/dashboards/ced_cluster_overview.json
+139
View File
@@ -0,0 +1,139 @@
{
"id": null,
"uid": "ced-cluster-overview",
"title": "Ced's K3s Cluster Overview",
"tags": [
"k3s",
"cluster",
"ced-noc"
],
"timezone": "browser",
"schemaVersion": 38,
"version": 1,
"refresh": "30s",
"panels": [
{
"type": "stat",
"title": "Total Nodes",
"id": 1,
"gridPos": {
"h": 4,
"w": 4,
"x": 0,
"y": 0
},
"targets": [
{
"expr": "count(kube_node_info)",
"refId": "A"
}
]
},
{
"type": "stat",
"title": "Ready Nodes",
"id": 2,
"gridPos": {
"h": 4,
"w": 4,
"x": 4,
"y": 0
},
"targets": [
{
"expr": "count(kube_node_status_condition{condition=\"Ready\",status=\"true\"})",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "Node CPU Usage (%)",
"id": 3,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 4
},
"targets": [
{
"expr": "100 - (avg by (instance) (irate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100)",
"legendFormat": "{{instance}}",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "Node Memory Usage (%)",
"id": 4,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 4
},
"targets": [
{
"expr": "(1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100",
"legendFormat": "{{instance}}",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "Pods per Namespace",
"id": 5,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 12
},
"targets": [
{
"expr": "count(kube_pod_info) by (namespace)",
"legendFormat": "{{namespace}}",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "API Server Request Rate",
"id": 6,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 12
},
"targets": [
{
"expr": "sum(rate(apiserver_request_total[5m])) by (verb)",
"legendFormat": "{{verb}}",
"refId": "A"
}
]
}
],
"templating": {
"list": [
{
"name": "namespace",
"type": "query",
"datasource": "Prometheus",
"query": "label_values(kube_pod_info, namespace)",
"current": {
"text": "All",
"value": ".*"
},
"includeAll": true,
"multi": true,
"regex": ""
}
]
}
}
k3s-homelab-full/dashboards/ced_ingress_metallb.json
+70
View File
@@ -0,0 +1,70 @@
{
"id": null,
"uid": "ced-ingress",
"title": "Ced's Ingress & MetalLB",
"tags": [
"ingress",
"metallb",
"ced-noc"
],
"timezone": "browser",
"schemaVersion": 38,
"version": 1,
"refresh": "30s",
"panels": [
{
"type": "graph",
"title": "NGINX Ingress HTTP Requests",
"id": 1,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 0
},
"targets": [
{
"expr": "sum(rate(nginx_ingress_controller_requests[5m])) by (ingress)",
"legendFormat": "{{ingress}}",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "NGINX Ingress 5xx Rate",
"id": 2,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 0
},
"targets": [
{
"expr": "sum(rate(nginx_ingress_controller_requests{status=~\"5..\"}[5m])) by (ingress)",
"legendFormat": "{{ingress}}",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "MetalLB Announced IPs",
"id": 3,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 8
},
"targets": [
{
"expr": "sum(metallb_speaker_announced)",
"legendFormat": "announced",
"refId": "A"
}
]
}
]
}
k3s-homelab-full/dashboards/ced_nodes_detail.json
+98
View File
@@ -0,0 +1,98 @@
{
"id": null,
"uid": "ced-node-detail",
"title": "Ced's K3s Nodes Detail",
"tags": [
"k3s",
"nodes",
"ced-noc"
],
"timezone": "browser",
"schemaVersion": 38,
"version": 1,
"refresh": "30s",
"templating": {
"list": [
{
"name": "node",
"type": "query",
"datasource": "Prometheus",
"query": "label_values(kube_node_info, node)",
"multi": false,
"includeAll": false
}
]
},
"panels": [
{
"type": "stat",
"title": "Node CPU Usage (%)",
"id": 1,
"gridPos": {
"h": 4,
"w": 6,
"x": 0,
"y": 0
},
"targets": [
{
"expr": "100 - (avg(irate(node_cpu_seconds_total{mode=\"idle\",instance=~\"$node\"}[5m])) * 100)",
"refId": "A"
}
]
},
{
"type": "stat",
"title": "Node Memory Usage (%)",
"id": 2,
"gridPos": {
"h": 4,
"w": 6,
"x": 6,
"y": 0
},
"targets": [
{
"expr": "(1 - (node_memory_MemAvailable_bytes{instance=~\"$node\"} / node_memory_MemTotal_bytes{instance=~\"$node\"})) * 100",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "Disk Usage (root fs)",
"id": 3,
"gridPos": {
"h": 8,
"w": 12,
"x": 0,
"y": 4
},
"targets": [
{
"expr": "1 - (node_filesystem_avail_bytes{mountpoint=\"/\",instance=~\"$node\"} / node_filesystem_size_bytes{mountpoint=\"/\",instance=~\"$node\"})",
"legendFormat": "rootfs",
"refId": "A"
}
]
},
{
"type": "graph",
"title": "Kubelet Pod Count",
"id": 4,
"gridPos": {
"h": 8,
"w": 12,
"x": 12,
"y": 4
},
"targets": [
{
"expr": "kubelet_running_pod_count{instance=~\"$node\"}",
"legendFormat": "{{instance}}",
"refId": "A"
}
]
}
]
}
k3s-homelab-full/diagrams/ced-k3s-from-text.txt
+41
View File
@@ -0,0 +1,41 @@
# Ced's K3s HomeLab (for draw.io text import)
# Paste this into draw.io via: Arrange → Insert → Advanced → From text
Ced HomeLab
VLAN 10.10.30.0/24
UniFi Dream Router (10.10.30.1)
Proxmox VE (10.10.30.250)
MetalLB VIPs (10.10.30.251-254)
Control Plane (Raspberry Pi)
k3s-django-1 (10.10.30.72) [control-plane, etcd]
k3s-django-2 (10.10.30.245) [control-plane, etcd]
k3s-django-3 (10.10.30.128) [control-plane, etcd]
Ingress Nodes
k3s-node-1 (10.10.30.219) [ingress-node=true]
k3s-node-2 (10.10.30.134) [ingress-node=true]
k3s-node-3 (10.10.30.222) [ingress-node=true]
Data Nodes
k3s-node-4 (10.10.30.126) [data-node=true]
k3s-node-5 (10.10.30.239) [data-node=true]
k3s-node-6 (10.10.30.208) [data-node=true]
Monitoring Nodes
k3s-node-7 (10.10.30.198) [monitoring-node=true]
k3s-node-8 (10.10.30.216) [monitoring-node=true]
k3s-node-9 (10.10.30.29) [monitoring-node=true]
Kubernetes Services
MetalLB (L2)
IPAddressPool: 10.10.30.251-10.10.30.254
ingress-nginx
Service: LoadBalancer → MetalLB VIP (e.g. 10.10.30.251)
kube-prometheus-stack (Ced's NOC)
Prometheus
Alertmanager
Grafana (LoadBalancer via MetalLB)
Demo nginx app
Ingress: demo.local → ingress-nginx → demo-nginx Service
k3s-homelab-full/docs/per-node-notes.md
+43
View File
@@ -0,0 +1,43 @@
# Ced's K3s Node Inventory
Use this file to track hardware details, OS, and roles for each node in the cluster.
## Control Plane Nodes
| Name | IP | Role | Hardware | OS / Version | Storage | Notes |
|---------------|-------------|------------------------------|--------------------|------------------------------|----------------|---------------------|
| k3s-django-1 | 10.10.30.72 | control-plane,etcd,master | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | SD / SSD | Primary API / etcd |
| k3s-django-2 | 10.10.30.245| control-plane,etcd,master | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | SD / SSD | |
| k3s-django-3 | 10.10.30.128| control-plane,etcd,master | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | SD / SSD | |
## Worker Nodes – Ingress
| Name | IP | Role | Hardware | OS / Version | Storage | Notes |
|------------|--------------|---------|--------------------|-----------------------------|---------|---------------------|
| k3s-node-1 | 10.10.30.219 | ingress | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
| k3s-node-2 | 10.10.30.134 | ingress | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
| k3s-node-3 | 10.10.30.222 | ingress | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
## Worker Nodes – Data
| Name | IP | Role | Hardware | OS / Version | Storage | Notes |
|------------|--------------|------|--------------------|-----------------------------|---------|---------------------|
| k3s-node-4 | 10.10.30.126 | data | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | e.g. databases |
| k3s-node-5 | 10.10.30.239 | data | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
| k3s-node-6 | 10.10.30.208 | data | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
## Worker Nodes – Monitoring
| Name | IP | Role | Hardware | OS / Version | Storage | Notes |
|------------|-------------|-------------|--------------------|-----------------------------|---------|-------------------------|
| k3s-node-7 | 10.10.30.198| monitoring | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | Prometheus, Loki, etc. |
| k3s-node-8 | 10.10.30.216| monitoring | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
| k3s-node-9 | 10.10.30.29 | monitoring | Raspberry Pi 4B ? | Debian 12 (Bookworm, ARM64) | | |
## Other Notes
- Proxmox VE: 10.10.30.250
- MetalLB VIP range: 10.10.30.251–10.10.30.254
- Ingress-nginx LB example: 10.10.30.251
Use this file to track future changes (upgrading SD → SSD, RAM sizes, Pi models, etc.).
k3s-homelab-full/kube-prom-values.yaml
+30
View File
@@ -0,0 +1,30 @@
prometheus:
prometheusSpec:
retention: 15d
scrapeInterval: 15s
nodeSelector:
monitoring-node: "true"
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 20Gi
alertmanager:
enabled: true
alertmanagerSpec:
nodeSelector:
monitoring-node: "true"
grafana:
enabled: true
adminPassword: "cedrocks"
service:
type: LoadBalancer
nodeSelector:
monitoring-node: "true"
persistence:
enabled: true
size: 10Gi
k3s-homelab-full/manifests/demo-app/demo-nginx.yaml
+54
View File
@@ -0,0 +1,54 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-nginx
labels:
app: demo-nginx
spec:
replicas: 3
selector:
matchLabels:
app: demo-nginx
template:
metadata:
labels:
app: demo-nginx
spec:
containers:
- name: nginx
image: nginx:stable
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: demo-nginx
spec:
selector:
app: demo-nginx
ports:
- port: 80
targetPort: 80
protocol: TCP
name: http
type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-nginx-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: demo.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: demo-nginx
port:
number: 80
k3s-homelab-full/manifests/ingress/grafana-ingress.yaml
+19
View File
@@ -0,0 +1,19 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: grafana.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ceds-noc-grafana
port:
number: 80
k3s-homelab-full/manifests/ingress/prometheus-ingress.yaml
+19
View File
@@ -0,0 +1,19 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: prometheus-ingress
namespace: monitoring
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: prometheus.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: ceds-noc-kube-prometheus-s-prometheus
port:
number: 9090
k3s-homelab-full/manifests/metallb/ipaddresspool.yaml
+8
View File
@@ -0,0 +1,8 @@
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: ceds-pool
namespace: metallb-system
spec:
addresses:
- 10.10.30.251-10.10.30.254
k3s-homelab-full/manifests/metallb/l2advertisement.yaml
+8
View File
@@ -0,0 +1,8 @@
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: ceds-l2
namespace: metallb-system
spec:
ipAddressPools:
- ceds-pool
k3s-homelab-full/scripts/03_label_nodes.sh
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[Label] Ingress nodes 1-3"
kubectl label node k3s-node-1 ingress-node=true --overwrite
kubectl label node k3s-node-2 ingress-node=true --overwrite
kubectl label node k3s-node-3 ingress-node=true --overwrite
echo "[Label] Data nodes 4-6"
kubectl label node k3s-node-4 data-node=true --overwrite
kubectl label node k3s-node-5 data-node=true --overwrite
kubectl label node k3s-node-6 data-node=true --overwrite
echo "[Label] Monitoring nodes 7-9"
kubectl label node k3s-node-7 monitoring-node=true --overwrite
kubectl label node k3s-node-8 monitoring-node=true --overwrite
kubectl label node k3s-node-9 monitoring-node=true --overwrite
kubectl get nodes -L ingress-node -L data-node -L monitoring-node
k3s-homelab-full/scripts/10_install_metallb.sh
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
NAMESPACE=metallb-system
echo "[MetalLB] Installing CRDs & core components..."
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml
echo "[MetalLB] Waiting for pods..."
kubectl rollout status -n ${NAMESPACE} deploy/controller --timeout=180s || true
echo "[MetalLB] Applying IPAddressPool and L2Advertisement (10.10.30.251-10.10.30.254)..."
kubectl apply -f manifests/metallb/ipaddresspool.yaml
kubectl apply -f manifests/metallb/l2advertisement.yaml
kubectl get ipaddresspools.metallb.io -n ${NAMESPACE}
kubectl get l2advertisements.metallb.io -n ${NAMESPACE}
k3s-homelab-full/scripts/20_install_ingress_nginx.sh
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
set -euo pipefail
NAMESPACE=ingress-nginx
echo "[Ingress] Adding Helm repo for ingress-nginx..."
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx >/dev/null 2>&1 || true
helm repo update
echo "[Ingress] Installing ingress-nginx (LoadBalancer, pinned to ingress-node=true)..."
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx --namespace ${NAMESPACE} --create-namespace --set controller.service.type=LoadBalancer --set controller.nodeSelector.ingress-node="true"
kubectl get pods -n ${NAMESPACE}
kubectl get svc -n ${NAMESPACE} ingress-nginx-controller
k3s-homelab-full/scripts/30_install_ceds_noc.sh
+17
View File
@@ -0,0 +1,17 @@
#!/usr/bin/env bash
set -euo pipefail
NAMESPACE=monitoring
echo "[NOC] Creating monitoring namespace (if not exists)..."
kubectl create namespace ${NAMESPACE} 2>/dev/null || true
echo "[NOC] Adding prometheus-community Helm repo..."
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts >/dev/null 2>&1 || true
helm repo update
echo "[NOC] Installing kube-prometheus-stack as 'ceds-noc'..."
helm upgrade --install ceds-noc prometheus-community/kube-prometheus-stack --namespace ${NAMESPACE} -f kube-prom-values.yaml
kubectl get pods -n ${NAMESPACE}
kubectl get svc -n ${NAMESPACE}