Fix deployment tooling: shell scripts and Terraform

Terraform (validated with the real terraform CLI - was never actually
run against this cluster, no state file existed):
- Delete main.tf: it declared a duplicate kubernetes_namespace.core
  (also in namespaces.tf) and a duplicate provider "kubernetes" block
  (also in providers.tf), both hard errors that would fail
  `terraform plan` immediately.
- Fix workloads.tf references to 6 files deleted in the manifest
  cleanup (jupiter-sts/svc, atlantis-sts/svc, fiesta-sts/svc) - now
  points at the canonical nearle-jupiter/atlantis/fiesta.yaml.
- Fix every kubernetes_manifest resource: they fed multi-document
  YAML (multiple '---'-separated docs per file) straight into
  yamldecode(), which only parses a single document. Rewrote using a
  split-on-'---' + for_each pattern, confirmed safe first by checking
  separator counts exactly match document counts for every affected
  file (no embedded '---' inside any script/config content).
- Add the doormile namespace; rename kubernetes_namespace to
  kubernetes_namespace_v1 (fixes a deprecation warning).
- `terraform validate` now passes clean.

Shell scripts:
- deploy-nearle-stack.sh only applied 4 of the ~13 files in
  manifests/nearle/ - missing the ConfigMap/Secrets fiesta/jupiter/
  titan/ariane need via envFrom, the fiesta gateway script ConfigMap,
  atlantis entirely, and the Gateway/ReferenceGrant/jupiter-cors-proxy
  resources. Now applies every file (verified by diffing the
  directory listing against the script).
- Added deploy-doormile.sh and deploy-ingress.sh - nothing previously
  applied ingress-unified.yaml or traefik-middlewares.yaml at all.
- Rewrote deploy.sh as an orchestrator calling all of the above in
  order (previously referenced a manifests/namespace.yaml layout that
  hasn't existed since before this repo's initial commit).
- Rewrote check-k8s-status.sh to check the real namespaces
  (core/nearle/alaska/doormile/kubernetes-dashboard) instead of a
  'nats-backend' namespace that never existed in this repo.
- Fixed a `cd` bug in setup-jetstream.sh that made it change into
  shfiles/ and then look for scripts/setup_jetstream.py there (a
  child directory that doesn't exist) - it could never have found its
  own target file. Now pulls NATS credentials from the live
  nats-credentials Secret instead of a third hardcoded copy.

Python scripts:
- sync_manifests.py had hardcoded Windows paths (e:\nats\kubernetes\...)
  - replaced with paths relative to the script's own location so it
  actually runs here (or anywhere). Verified by running it.
- setup_jetstream.py created durable consumers under different names
  than worker.py computes at runtime ({NATS_CONSUMER}_{subject}), so
  its max_deliver/ack_wait settings never actually reached the
  consumers workers bind to. Naming now derived with the same logic
  worker.py uses - verified all 10 derived names match workers.yaml
  exactly.
- purge-old-messages.py had hardcoded NATS credentials with no env
  var override at all - fixed to match the pattern used everywhere
  else.
This commit is contained in:
Suriya
2026-07-18 16:08:15 +05:30
parent 91dd240431
commit 0a8c3b0374
14 changed files with 303 additions and 270 deletions

View File

@@ -1,9 +1,13 @@
#!/bin/bash
# Setup JetStream stream and consumer for NATS
set -euo pipefail
echo "🚀 Setting up NATS JetStream..."
cd "$(dirname "$0")"
# scripts/ is a sibling of shfiles/, both under the repo root - cd there so
# the relative path below resolves regardless of where this is invoked from.
cd "$(dirname "$0")/.."
# Check if Python is available
if ! command -v python3 &> /dev/null; then
@@ -17,11 +21,19 @@ if ! python3 -c "import nats" 2>/dev/null; then
pip3 install nats-py
fi
# Set environment variables
export NATS_URL="nats://nats.workolik.com:4222"
export NATS_USER="admin"
export NATS_PASSWORD="package@321#"
# Pull live credentials from the cluster's Secret instead of hardcoding them
# here - avoids yet another copy of the password to keep in sync if rotated.
if command -v kubectl &> /dev/null && kubectl get secret nats-credentials -n core &> /dev/null 2>&1; then
export NATS_USER
NATS_USER=$(kubectl get secret nats-credentials -n core -o jsonpath='{.data.username}' | base64 -d)
export NATS_PASSWORD
NATS_PASSWORD=$(kubectl get secret nats-credentials -n core -o jsonpath='{.data.password}' | base64 -d)
else
echo "⚠️ Could not read nats-credentials Secret from the cluster (kubectl not available or not connected)."
echo " Set NATS_USER / NATS_PASSWORD yourself before running this script."
fi
export NATS_URL="${NATS_URL:-nats://66.116.226.161:4222}"
# Run the setup script
python3 scripts/setup_jetstream.py