How to Debug a Kubernetes Service That Is Not Reachable

A Kubernetes Service can fail at several layers: the client may resolve the wrong name, the Service may select no ready pods, port and targetPort may not match, the application may listen on the wrong interface, or network policy and node networking may block traffic. Test one layer at a time instead of recreating the Service immediately.

Define where the request fails

Record whether the client is:

• another pod in the same namespace
• a pod in another namespace
• a node
• outside the cluster through an Ingress or LoadBalancer

The valid DNS name and network path differ for each client. Start a temporary diagnostic pod only if your cluster policy permits the image:

kubectl run network-debug --rm -it --restart=Never \
  --image=busybox:1.36 -n NAMESPACE -- sh

Avoid attaching a shell to a production application container when an isolated diagnostic pod is sufficient.

Check Service DNS and ClusterIP

From a pod in the cluster:

nslookup SERVICE
nslookup SERVICE.NAMESPACE.svc.cluster.local

Inspect the Service itself:

kubectl get service SERVICE -n NAMESPACE -o wide
kubectl describe service SERVICE -n NAMESPACE

A headless Service has clusterIP: None and resolves differently from a normal ClusterIP Service. ExternalName also behaves differently and has no pod endpoints.

Verify ports and target ports

kubectl get service SERVICE -n NAMESPACE -o yaml
kubectl get pods -n NAMESPACE -l 'SELECTOR' -o wide

Check:

• Service port: the port clients use
• targetPort: the destination port or named container port
• protocol: normally TCP or UDP
• selectors: labels that choose backend pods

A named targetPort must match a named port on the selected pods. A numeric containerPort declaration is descriptive and does not make an application listen by itself.

Inspect EndpointSlices

kubectl get endpointslice -n NAMESPACE \
  -l kubernetes.io/service-name=SERVICE -o wide
kubectl describe endpointslice -n NAMESPACE \
  -l kubernetes.io/service-name=SERVICE

If there are no endpoints, compare the Service selector with pod labels. If endpoints exist but are not ready, inspect pod readiness conditions and probes.

For Services without selectors, EndpointSlices must be managed separately. Do not add selectors unless that matches the intended design.

Test the backend directly

From an approved diagnostic pod, test both the Service and a backend pod IP:

wget -S -O- http://SERVICE:NAMED_PORT/
wget -S -O- http://POD_IP:TARGET_PORT/

If the pod IP fails, inspect whether the application is running and listening on the pod interface:

kubectl logs POD -n NAMESPACE --tail=200
kubectl exec POD -n NAMESPACE -- sh -c 'cat /proc/net/tcp /proc/net/tcp6'

An application bound only to 127.0.0.1 cannot accept traffic sent to the pod IP.

Check readiness and NetworkPolicies

kubectl describe pod POD -n NAMESPACE
kubectl get networkpolicy -A

Only ready endpoints normally receive Service traffic. Confirm that NetworkPolicies allow the client’s namespace or pod labels, the destination pods, and the correct port. Remember that enforcement depends on the installed CNI.

Investigate the node layer last

If DNS, Service configuration, endpoints, direct pod access, and policy are correct, compare behavior from different nodes. Inspect kube-proxy or the CNI’s service implementation and logs according to the cluster design.

Do not flush iptables, replace CNI state, or restart networking components across the cluster as an initial diagnostic step.

Diagnose it with Rumus

Diagnose why this Kubernetes Service is unreachable. Establish the client location
and exact error; inspect DNS, Service type and ports, selectors, EndpointSlices,
pod readiness and listeners, direct pod connectivity, NetworkPolicies, and node
distribution. Then inspect kube-proxy or CNI state only if the earlier layers pass.
Start read-only and do not recreate resources or restart networking without approval.

Confirm the fix

Repeat tests from the original client and an independent diagnostic pod. Verify the Service name, ClusterIP, and more than one backend endpoint where applicable. Also confirm that the fix does not bypass intended NetworkPolicy or expose the workload beyond its required scope.