1. Introduction
In this tutorial, we’ll learn how to fix the error “Connection to server was refused” when using kubectl. First, let’s understand how kubectl communicates with the Kubernetes API server and why this error can occur. Finally, we’ll examine two common root causes for this error and see how we can resolve them.
2. Communication Between kubectl and the Kubernetes API Server
kubectl is a command line tool that allows us to manage Kubernetes clusters. It does this by authenticating with the Kubernetes API server of the cluster and making API calls to perform a range of management actions.
When kubectl is unable to communicate with the Kubernetes API server for some reason, it produces an error that mentions something like “Connection to server was refused”. For example, by running a simple command to list pods, we can get the following output:
$ kubectl get pods
The connection to the server 192.168.122.54:6443 was refused - did you specify the right host or port?
Next, we’ll look at the two most common causes of this type of error when using kubectl and how to solve them.
3. Configuration Issue in kubectl
A very common root cause for the problem of kubectl not being able to communicate with the API server is a configuration mistake. Basically, kubectl needs a kubeconfig file to indicate how to reach the API server and manage the Kubernetes cluster.
Therefore, a common reason for such an error is a configuration mistake in the kubeconfig file. However, this error can also arise if the kubeconfig file doesn’t exist.
To check whether the error results from the absence of the kubeconfig file, we can manually look for a file called config in the ~/.kube/ directory. Alternatively, we can check this via the kubectl command line:
$ kubectl config view
apiVersion: v1
clusters: null
contexts: null
current-context: ""
kind: Config
preferences: {}
users: null
The output of the above command indicates that the kubeconfig file (~/.kube/config) doesn’t exist, since the parameters aren’t specified (values are null or “”).
In this case, we can proceed as follows to create the missing file properly:
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config
Next, when we run the kubectl config view command again, we can see that the parameters are set:
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://192.168.122.54:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: DATA+OMITTED
client-key-data: DATA+OMITTED
Now, we just need to check that the configuration parameters are correct. In particular, we need to ensure that the value of the server parameter matches the API server’s actual IP address and port number.
4. Kubernetes API Server is Not Running
Once we’ve confirmed that kubectl is trying to access the API server via the proper IP address and port, the most likely cause is that the server isn’t running.
The Kubernetes API server runs as a container (kube-apiserver) within pods in the kube-system namespace and automatically starts along with the other Kubernetes services. Therefore, a change in the environment usually causes this issue, preventing the Kubernetes services from initializing.
A very common case occurs when we enable the swap memory on Kubernetes nodes. This is because, by default, Kubernetes nodes don’t support swap memory management.
So, an easy way to solve this problem is to disable swap. We can disable the swap memory instantly using the following command:
$ sudo swapoff -a
However, to make this permanent we need to edit the /etc/fstab file, commenting out the line referring to swap memory. In other words, we need to add # to the beginning of the line that starts with /swapfile.
If swap memory needs to be enabled, we need to enable a feature called NodeSwap on each node in the Kubernetes cluster. We recommend following the official Kubernetes documentation regarding swap memory management support. We also need to take into account the associated implications since this changes Kubernetes’ default behavior.
5. Conclusion
In this tutorial, we studied how to solve the error “Connection to server was refused” when using kubectl. As we’ve learned, this error usually occurs in two situations: when kubectl has incorrect information in its configuration about the Kubernetes API server, or the Kubernetes API server isn’t running.