# Kubernetes用Helm安装Ingress并踩一下使用的坑

# 前言

IngressKubernetes一个非常重要的Controller,它类似一个路由转发的组件,可以让外界访问Kubernetes内部的Service。除了Ingress,还有NodePortLoadBalance等方式,Ingress暴露给外界的方式还是很常用的。

# 2 安装Ingress

我们通过helm来安装,会方便一些,先更新helm的仓库。

$ helm repo update

Helm相关文章:用Helm部署Kubernetes应用,支持多环境部署与版本回滚 (opens new window)

更新完之后,查找仓库关于Ingress的包有哪些:

$ $ helm search repo ingress
NAME                          	CHART VERSION	APP VERSION	DESCRIPTION                                       
azure/gce-ingress             	1.2.0        	1.4.0      	A GCE Ingress Controller                          
azure/ingressmonitorcontroller	1.0.48       	1.0.47     	IngressMonitorController chart that runs on kub...
azure/nginx-ingress           	1.41.2       	v0.34.1    	An nginx Ingress controller that uses ConfigMap...
stable/nginx-ingress          	0.9.5        	0.10.2     	An nginx Ingress controller that uses ConfigMap...
azure/contour                 	0.2.0        	v0.15.0    	Contour Ingress controller for Kubernetes         
azure/external-dns            	1.8.0        	0.5.14     	Configure external DNS servers (AWS Route53, Go...
azure/kong                    	0.36.7       	1.4        	DEPRECATED The Cloud-Native Ingress and API-man...
azure/lamp                    	1.1.3        	7          	Modular and transparent LAMP stack chart suppor...
azure/nginx-lego              	0.3.1        	           	Chart for nginx-ingress-controller and kube-lego  
azure/traefik                 	1.87.2       	1.7.24     	A Traefik based Kubernetes ingress controller w...
azure/voyager                 	3.2.4        	6.0.0      	DEPRECATED Voyager by AppsCode - Secure Ingress...
stable/external-dns           	0.4.9        	0.4.8      	Configure external DNS servers (AWS Route53, Go...
stable/lamp                   	0.1.4        	           	Modular and transparent LAMP stack chart suppor...
stable/nginx-lego             	0.3.1        	           	Chart for nginx-ingress-controller and kube-lego  
stable/traefik                	1.24.1       	1.5.3      	A Traefik based Kubernetes ingress controller w...
stable/voyager                	3.1.0        	6.0.0-rc.0 	Voyager by AppsCode - Secure Ingress Controller...

选择azure/nginx-ingress来安装,注意是有版本的。安装如下:

$ helm install pkslow-ingress azure/nginx-ingress

安装成功后,控制台会有输出相关的使用说明。但要注意的是,要去Kubernetes Dashboard查看一下是否真的安装成功。我安装遇到过失败,原因都是因为镜像下载失败。解决方案是打开全局代理,先手动下载好相关镜像。如:

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1
k8s.gcr.io/defaultbackend-amd64:1.5
jettech/kube-webhook-certgen:v1.0.0
quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0

之后就安装成功了,相关的Pods都跑起来了。相关的Deployment有:

# 3 使用Ingress

# 3.1 访问一个服务

一个最简单的例子如下:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.19.0
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginx-service
spec:
  ports:
    - port: 80
      name: nginx-service
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost

这样配置后,当我们访问http://localhost/时,就会把我们的请求转发到nginx-service80端口上去。如下所示:

# 3.2 访问多个服务

当要访问多个服务时,事情就变得复杂起来了。访问多个服务,有两种配置方式,一种是通过URL路径匹配再转发,另一种是通过子域名转发。

# 3.2.1 子域名方式

通过子域名转发如下配置:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: nginx.localhost
    - http:
        paths:
          - path: /
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: springboot.localhost

为了节省篇幅,这里就只展示Ingress的配置了。

访问http://nginx.localhost/如下:

访问http://springboot.localhost/swagger-ui.html如下,注意这个URL带了子路径swagger-ui.html

# 3.2.2 URL路径匹配方式

那通过URL路径匹配方式是不是这样配置呢?

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /nginx
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

这样配置后,会直接报404,但不是Ingress404,而是NginxSpringboot404。说明请求已经成功转发到对应的service了,但路径有问题。原因是,当这样配置时,Ingress会把path也转发到service上。所以实际效果如下:

localhost/nginx      --> nginx-service/nginx
localhost/springboot --> springboot-service/springboot

使用子域名没有问题,那是因为路径是/

解决方案一:配置rewrite,如nginx.ingress.kubernetes.io/rewrite-target: /,具体如下:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - http:
        paths:
          - path: /nginx
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

这样配置后,就变成了:

localhost/nginx      --> nginx-service/
localhost/springboot --> springboot-service/

这样似乎是解决了问题,但实际并没有,比如访问localhost/springboot可以正常访问:

但访问http://localhost/springboot/swagger-ui.html,还是转发到了/上:

解决方案二:使用正则rewrite,如下:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - http:
        paths:
          - path: /nginx(/|$)(.*)
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot(/|$)(.*)
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

可正常访问带子路径了:http://localhost/springboot/swagger-ui.html#/base-controller/pkslowUsingGET

注:使用正则rewrite时,下面这样也是可以的:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - http:
        paths:
          - path: /nginx/(.*)
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot/(.*)
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

除了一个小问题:http://localhost/nginx无法正常访问,要http://localhost/nginx/才可以,因为少了/会匹配不到,返回Ingress404

# 4 总结

过了一遍,坑真不少。使用子域名感觉是比较好的方式。另外,Ingress还有一个坑,它是实现HTTP/HTTPS转发的,但TCP就不行了,比如我在Kubernetes安装了一个MySQL数据库,需要把地址和3306TCP方式暴露给外面,就比较麻烦了,我们后续再讨论吧。

上次更新: 2023/8/18 23:39:36