Kubernetes Broken Due To Unknown Reasons

Problem 1: Admin User Unable to Login to Cluster via Controller (Master Node)

# SSL Error:
The connection to the server x.x.x.x:6443 was refused - did you specify the right host or port?

# Resolution to the SSL problem:
sudo -i
swapoff -a
exit
strace -eopenat kubectl version

# User privilege error:
kim@controller01:~$ kubectl cluster-info
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
error: You must be logged in to the server (Unauthorized)

kim@controller01:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.10", GitCommit:"8152330a2b6ca3621196e62966ef761b8f5a61bb", GitTreeState:"clean", BuildDate:"2021-08-11T18:06:15Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
error: You must be logged in to the server (the server has asked for the client to provide credentials)

# Resolution to Admin user privilege error:
# Grant current user admin privileges on Kubernetes
# mkdir -p $HOME/.kube # this was done during previous setup
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config # this was done during previous setup

Problem 2: SSL Certificates Not Automatically Renewed

# Error message with kimconnect.com SSL Cert
Warning  Failed   84m (x328 over 13d)  cert-manager  The certificate request has failed to complete and will be retried: Failed to wait for order resource "kimconnect-cert-qlnl9-1800784958" to become ready: order is in "invalid" state:

kim@controller01:~$ k get certificaterequests.cert-manager.io
NAME                          READY   AGE
kimconnect-cert-jqlvf         True    90d
kimconnect-cert-qlnl9         False   30d

# Try to delete cert requests and secrets, and wait for cert to regenerate
k delete certificaterequests kimconnect-cert-qlnl9
k delete secret kimconnect-cert

# Try to force cert to renew before 1440 hours (immediately)
kubectl patch certificate kimconnect-cert --patch '
- op: replace
  path: /spec/renewBefore
  value: 1440h
' --type=json

# Wait for cert to become ready, then reverse the change
kubectl patch certificate kimconnect-cert --patch '
- op: remove
  path: /spec/renewBefore
' --type=json

# Worst case scenario, delete the cert and recreate it
k delete cert kimconnect-cert
cat <<EOF > kimconnect-cert.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kimconnect-cert
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
    acme.cert-manager.io/http01-edit-in-place: "true"
    kubernetes.io/tls-acme: "true"
spec:
  dnsNames:
    - kimconnect.com
    - www.kimconnect.com
  secretName: kimconnect-cert
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
EOF
kubectl apply -f kimconnect-cert.yaml

Leave a Reply

Your email address will not be published. Required fields are marked *