# Kubernetes用Helm安装Ingress并踩一下使用的坑

# 前言

Ingress是Kubernetes一个非常重要的Controller，它类似一个路由转发的组件，可以让外界访问Kubernetes内部的Service。除了Ingress，还有NodePort、LoadBalance等方式，Ingress暴露给外界的方式还是很常用的。

# 2 安装Ingress

我们通过helm来安装，会方便一些，先更新helm的仓库。

$ helm repo update

更新完之后，查找仓库关于Ingress的包有哪些：

$ $ helm search repo ingress
NAME                          	CHART VERSION	APP VERSION	DESCRIPTION                                       
azure/gce-ingress             	1.2.0        	1.4.0      	A GCE Ingress Controller                          
azure/ingressmonitorcontroller	1.0.48       	1.0.47     	IngressMonitorController chart that runs on kub...
azure/nginx-ingress           	1.41.2       	v0.34.1    	An nginx Ingress controller that uses ConfigMap...
stable/nginx-ingress          	0.9.5        	0.10.2     	An nginx Ingress controller that uses ConfigMap...
azure/contour                 	0.2.0        	v0.15.0    	Contour Ingress controller for Kubernetes         
azure/external-dns            	1.8.0        	0.5.14     	Configure external DNS servers (AWS Route53, Go...
azure/kong                    	0.36.7       	1.4        	DEPRECATED The Cloud-Native Ingress and API-man...
azure/lamp                    	1.1.3        	7          	Modular and transparent LAMP stack chart suppor...
azure/nginx-lego              	0.3.1        	           	Chart for nginx-ingress-controller and kube-lego  
azure/traefik                 	1.87.2       	1.7.24     	A Traefik based Kubernetes ingress controller w...
azure/voyager                 	3.2.4        	6.0.0      	DEPRECATED Voyager by AppsCode - Secure Ingress...
stable/external-dns           	0.4.9        	0.4.8      	Configure external DNS servers (AWS Route53, Go...
stable/lamp                   	0.1.4        	           	Modular and transparent LAMP stack chart suppor...
stable/nginx-lego             	0.3.1        	           	Chart for nginx-ingress-controller and kube-lego  
stable/traefik                	1.24.1       	1.5.3      	A Traefik based Kubernetes ingress controller w...
stable/voyager                	3.1.0        	6.0.0-rc.0 	Voyager by AppsCode - Secure Ingress Controller...

选择azure/nginx-ingress来安装，注意是有版本的。安装如下：

$ helm install pkslow-ingress azure/nginx-ingress

安装成功后，控制台会有输出相关的使用说明。但要注意的是，要去Kubernetes Dashboard查看一下是否真的安装成功。我安装遇到过失败，原因都是因为镜像下载失败。解决方案是打开全局代理，先手动下载好相关镜像。如：

us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1
k8s.gcr.io/defaultbackend-amd64:1.5
jettech/kube-webhook-certgen:v1.0.0
quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0

之后就安装成功了，相关的Pods都跑起来了。相关的Deployment有：

# 3 使用Ingress

# 3.1 访问一个服务

一个最简单的例子如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.19.0
          ports:
            - containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginx-service
spec:
  ports:
    - port: 80
      name: nginx-service
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost

这样配置后，当我们访问http://localhost/时，就会把我们的请求转发到nginx-service的80端口上去。如下所示：

# 3.2 访问多个服务

当要访问多个服务时，事情就变得复杂起来了。访问多个服务，有两种配置方式，一种是通过URL路径匹配再转发，另一种是通过子域名转发。

# 3.2.1 子域名方式

通过子域名转发如下配置：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: nginx.localhost
    - http:
        paths:
          - path: /
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: springboot.localhost

为了节省篇幅，这里就只展示Ingress的配置了。

访问http://nginx.localhost/如下：

访问http://springboot.localhost/swagger-ui.html如下，注意这个URL带了子路径swagger-ui.html：

# 3.2.2 URL路径匹配方式

那通过URL路径匹配方式是不是这样配置呢？

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /nginx
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

这样配置后，会直接报404，但不是Ingress的404，而是Nginx或Springboot的404。说明请求已经成功转发到对应的service了，但路径有问题。原因是，当这样配置时，Ingress会把path也转发到service上。所以实际效果如下：

localhost/nginx      --> nginx-service/nginx
localhost/springboot --> springboot-service/springboot

使用子域名没有问题，那是因为路径是/。

解决方案一：配置rewrite，如nginx.ingress.kubernetes.io/rewrite-target: /，具体如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - http:
        paths:
          - path: /nginx
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

这样配置后，就变成了：

localhost/nginx      --> nginx-service/
localhost/springboot --> springboot-service/

这样似乎是解决了问题，但实际并没有，比如访问localhost/springboot可以正常访问：

但访问http://localhost/springboot/swagger-ui.html，还是转发到了/上：

解决方案二：使用正则rewrite，如下：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - http:
        paths:
          - path: /nginx(/|$)(.*)
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot(/|$)(.*)
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

可正常访问带子路径了：http://localhost/springboot/swagger-ui.html#/base-controller/pkslowUsingGET

注：使用正则rewrite时，下面这样也是可以的：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - http:
        paths:
          - path: /nginx/(.*)
            backend:
              serviceName: nginx-service
              servicePort: 80
      host: localhost
    - http:
        paths:
          - path: /springboot/(.*)
            backend:
              serviceName: springboot-service
              servicePort: 8080
      host: localhost

除了一个小问题：http://localhost/nginx无法正常访问，要http://localhost/nginx/才可以，因为少了/会匹配不到，返回Ingress的404。

# 4 总结

过了一遍，坑真不少。使用子域名感觉是比较好的方式。另外，Ingress还有一个坑，它是实现HTTP/HTTPS转发的，但TCP就不行了，比如我在Kubernetes安装了一个MySQL数据库，需要把地址和3306以TCP方式暴露给外面，就比较麻烦了，我们后续再讨论吧。

← 把Spring Cloud Data Flow部署在Kubernetes上，再跑个任务试试 Kubernetes备份所有集群中的资源为yaml文件 →