一. 官方获取x_forwarded_for的方法

https://doc.traefik.io/traefik/routing/entrypoints/#forwarded-headers

添加环境变量 vi /var/lib/rancher/k3s/server/manifests/traefik-config.yaml

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    additionalArguments:
      - "--entryPoints.web.proxyProtocol.insecure"
      - "--entryPoints.web.forwardedHeaders.insecure"
      - "--entryPoints.websecure.proxyProtocol.insecure"
      - "--entryPoints.websecure.forwardedHeaders.insecure"
      - "--entryPoints.web.forwardedHeaders.trustedIPs=127.0.0.1/32,10.0.0.0/8,172.16.0.0/16"
      - "--entryPoints.websecure.forwardedHeaders.trustedIPs=127.0.0.1/32,10.0.0.0/8,172.16.0.0/16"

systemctl daemon-reload
systemctl restart k3s

• 我在阿里云上部署的集群，这一步操作只能获取负载均衡传进来的x-real-ip，自己调用的ip地获取不到

二. externalTrafficPolicy修改为local

添加第1步的文件内容

      ...
    #注意这里是一级
    service:
      spec:
        externalTrafficPolicy: Local

重启k3s

• 如果还未生效, 添加ipv4的转发功能

echo net.ipv4.ip_forward=1 >> /etc/sysctl.conf
sysctl -p

# 这一步操作阿里云就可以获取真实的ip了

三. 关闭flannel的ip-masq (未测)

vi /run/flannel/subnet.env
FLANNEL_IPMASQ=false

四、安装k3s时不安装flannel (未测)

安装后，修改flannel的yaml文件，手动安装

#安装k3s不安装插件
curl -sfL https://get.k3s.io |   sh -s - server --flannel-backend=none --no-deploy=servicelb --disable=traefik --disable servicelb


#下载kube-flannel.yml 添加ip-masq=false 再安装
#https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl appyle -f kube-flannel.yml

#安装traefik
helm install traefik traefik/traefik


修改traefik的externalTrafficPolicy为:local
kubectl edit svc/traefik 
  externalIPs: 
  - 123.123.123.123
  externalTrafficPolicy: Local