Kubernetes ingress-nginx slow on baremetal - November 2018
What’s been going on?
A Python project, some quoting projects, back to scala dev, focus on Kafka, then Docker, then Kubernetes, and finally ingress. So no posts for ages.
What’s ingress?
If you want to expose an akk.http end point from your image, to your LAN then you need an ingress.
What’s nginx ingress?
Its one of the ways to do it!
What’s the problem?
I have a small home multi-server kubernetes bare metal set up. I also have event driven reactive kafka microservices chatting away, all akka based scala microservices.
I want a health service to capture some metrics (yes I did set up ELK, but for now its turned off). So, I have a couple of them running consuming health metrics, publishing to a Rest end point.
I can curl internally, then set up ingress-nginx. It ran like a dog - over 10 seconds per upstream request. I got into editing the nginx config map to add debug logging (- –v=5).
You can see the request upstream time in the controller logs, so do google that.
Anyway, after several attempts, I simply removed it all!
To remove ingress-nginx
kubectl get service --all-namespaces kubectl get pod --all-namespaces kubectl get ingress --all-namespaces kubectl get deploy --all-namespaces
Then run the delete for everything that looks like nginx.
Clean install
Do this for baremetal, ie google for the instructions for ingress-nginx baremetal. There is a command at the top of the page, and then one further down.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
Then expose your service, and finally configure the ingress:
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: jpghealth-ingress namespace: default annotations: nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/ssl-redirect: "false" spec: rules: - http: paths: - path: /jpghealth backend: serviceName: jpghealth servicePort: 8080
and create the ingress
kubectl create -f ingress.yaml
Accessing the web page
Then get the nodeport port on the host its running on
kubectl get service -o wide --all-namespaces
You will see the ingress-nginx Nodeport has a port listed like 80:32446, so now I can browse to it at http://192.168.1.245:32446/jpghealth/index.html
My health page loads, and starts up a restfull poll back to the service, giving me my health information.
Why write all this stuff?
Basically, the first time I installed ingress-nginx I messed up. Nothing on the web described it running with over 10 seconds latency. So, maybe this may happen to you.
The solution? Start again!