前言

K8s + SpringBoot實(shí)現(xiàn)零宕機(jī)發(fā)布：健康檢查+滾動(dòng)更新+優(yōu)雅停機(jī)+彈性伸縮+Prometheus監(jiān)控+配置分離（鏡像復(fù)用）。

基于 Spring Boot + MyBatis Plus + Vue & Element 實(shí)現(xiàn)的后臺(tái)管理系統(tǒng) + 用戶(hù)小程序，支持 RBAC 動(dòng)態(tài)權(quán)限、多租戶(hù)、數(shù)據(jù)權(quán)限、工作流、三方登錄、支付、短信、商城等功能。

項(xiàng)目地址：https://github.com/YunaiV/ruoyi-vue-pro

配置

健康檢查

健康檢查類(lèi)型：就緒探針（readiness）+ 存活探針（liveness）

探針類(lèi)型：exec（進(jìn)入容器執(zhí)行腳本）、tcpSocket（探測(cè)端口）、httpGet（調(diào)用接口）

業(yè)務(wù)層面

項(xiàng)目依賴(lài) pom.xml

org.springframework.boot
spring-boot-starter-actuator

定義訪問(wèn)端口、路徑及權(quán)限 application.yaml

management:
server:
port:50000#啟用獨(dú)立運(yùn)維端口
endpoint:#開(kāi)啟health端點(diǎn)
health:
probes:
enabled:true
endpoints:
web:
exposure:
base-path:/actuator#指定上下文路徑，啟用相應(yīng)端點(diǎn)
include:health

將暴露/actuator/health/readiness和/actuator/health/liveness兩個(gè)接口，訪問(wèn)方式如下：

http://127.0.0.1:50000/actuator/health/readiness
http://127.0.0.1:50000/actuator/health/liveness

運(yùn)維層面

k8s部署模版deployment.yaml

apiVersion:apps/v1
kind:Deployment
spec:
template:
spec:
containers:
-name:{APP_NAME}
image:{IMAGE_URL}
imagePullPolicy:Always
ports:
-containerPort:{APP_PORT}
-name:management-port
containerPort:50000#應(yīng)用管理端口
readinessProbe:#就緒探針
httpGet:
path:/actuator/health/readiness
port:management-port
initialDelaySeconds:30#延遲加載時(shí)間
periodSeconds:10#重試時(shí)間間隔
timeoutSeconds:1#超時(shí)時(shí)間設(shè)置
successThreshold:1#健康閾值
failureThreshold:6#不健康閾值
livenessProbe:#存活探針
httpGet:
path:/actuator/health/liveness
port:management-port
initialDelaySeconds:30#延遲加載時(shí)間
periodSeconds:10#重試時(shí)間間隔
timeoutSeconds:1#超時(shí)時(shí)間設(shè)置
successThreshold:1#健康閾值
failureThreshold:6#不健康閾值

滾動(dòng)更新

k8s資源調(diào)度之滾動(dòng)更新策略，若要實(shí)現(xiàn)零宕機(jī)發(fā)布，需支持健康檢查

apiVersion:apps/v1
kind:Deployment
metadata:
name:{APP_NAME}
labels:
app:{APP_NAME}
spec:
selector:
matchLabels:
app:{APP_NAME}
replicas:{REPLICAS}#Pod副本數(shù)
strategy:
type:RollingUpdate#滾動(dòng)更新策略
rollingUpdate:
maxSurge:1#升級(jí)過(guò)程中最多可以比原先設(shè)置的副本數(shù)多出的數(shù)量
maxUnavailable:1#升級(jí)過(guò)程中最多有多少個(gè)POD處于無(wú)法提供服務(wù)的狀態(tài)

優(yōu)雅停機(jī)

在K8s中，當(dāng)我們實(shí)現(xiàn)滾動(dòng)升級(jí)之前，務(wù)必要實(shí)現(xiàn)應(yīng)用級(jí)別的優(yōu)雅停機(jī)。否則滾動(dòng)升級(jí)時(shí)，還是會(huì)影響到業(yè)務(wù)。使應(yīng)用關(guān)閉線程、釋放連接資源后再停止服務(wù)

業(yè)務(wù)層面

項(xiàng)目依賴(lài) pom.xml

org.springframework.boot
spring-boot-starter-actuator

定義訪問(wèn)端口、路徑及權(quán)限 application.yaml

spring:
application:
name:
profiles:
active:@profileActive@
lifecycle:
timeout-per-shutdown-phase:30s#停機(jī)過(guò)程超時(shí)時(shí)長(zhǎng)設(shè)置30s，超過(guò)30s，直接停機(jī)

server:
port:8080
shutdown:graceful#默認(rèn)為IMMEDIATE，表示立即關(guān)機(jī)；GRACEFUL表示優(yōu)雅關(guān)機(jī)

management:
server:
port:50000#啟用獨(dú)立運(yùn)維端口
endpoint:#開(kāi)啟shutdown和health端點(diǎn)
shutdown:
enabled:true
health:
probes:
enabled:true
endpoints:
web:
exposure:
base-path:/actuator#指定上下文路徑，啟用相應(yīng)端點(diǎn)
include:health,shutdown

將暴露/actuator/shutdown接口，調(diào)用方式如下：

curl-XPOST127.0.0.1:50000/actuator/shutdown

運(yùn)維層面

確保dockerfile模版集成curl工具，否則無(wú)法使用curl命令

FROMopenjdk:8-jdk-alpine
#構(gòu)建參數(shù)
ARGJAR_FILE
ARGWORK_PATH="/app"
ARGEXPOSE_PORT=8080

#環(huán)境變量
ENVJAVA_OPTS=""
JAR_FILE=${JAR_FILE}

#設(shè)置時(shí)區(qū)
RUNln-sf/usr/share/zoneinfo/Asia/Shanghai/etc/localtime&&echo'Asia/Shanghai'>/etc/timezone
RUNsed-i's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g'/etc/apk/repositories
&&apkadd--no-cachecurl
#將maven目錄的jar包拷貝到docker中，并命名為for_docker.jar
COPYtarget/$JAR_FILE$WORK_PATH/


#設(shè)置工作目錄
WORKDIR$WORK_PATH



>基于SpringCloudAlibaba+Gateway+Nacos+RocketMQ+Vue&Element實(shí)現(xiàn)的后臺(tái)管理系統(tǒng)+用戶(hù)小程序，支持RBAC動(dòng)態(tài)權(quán)限、多租戶(hù)、數(shù)據(jù)權(quán)限、工作流、三方登錄、支付、短信、商城等功能
>
>*項(xiàng)目地址：
>*視頻教程：

#指定于外界交互的端口
EXPOSE$EXPOSE_PORT
#配置容器，使其可執(zhí)行化
ENTRYPOINTexecjava$JAVA_OPTS-jar$JAR_FILE

k8s部署模版deployment.yaml

注：經(jīng)驗(yàn)證，java項(xiàng)目可省略結(jié)束回調(diào)鉤子的配置

此外，若需使用回調(diào)鉤子，需保證鏡像中包含curl工具，且需注意應(yīng)用管理端口（50000）不能暴露到公網(wǎng)

apiVersion:apps/v1
kind:Deployment
spec:
template:
spec:
containers:
-name:{APP_NAME}
image:{IMAGE_URL}
imagePullPolicy:Always
ports:
-containerPort:{APP_PORT}
-containerPort:50000
lifecycle:
preStop:#結(jié)束回調(diào)鉤子
exec:
command:["curl","-XPOST","127.0.0.1:50000/actuator/shutdown"]

彈性伸縮

為pod設(shè)置資源限制后，創(chuàng)建HPA

apiVersion:apps/v1
kind:Deployment
metadata:
name:{APP_NAME}
labels:
app:{APP_NAME}
spec:
template:
spec:
containers:
-name:{APP_NAME}
image:{IMAGE_URL}
imagePullPolicy:Always
resources:#容器資源管理
limits:#資源限制（監(jiān)控使用情況）
cpu:0.5
memory:1Gi
requests:#最小可用資源（靈活調(diào)度）
cpu:0.15
memory:300Mi
---
kind:HorizontalPodAutoscaler#彈性伸縮控制器
apiVersion:autoscaling/v2beta2
metadata:
name:{APP_NAME}
spec:
scaleTargetRef:
apiVersion:apps/v1
kind:Deployment
name:{APP_NAME}
minReplicas:{REPLICAS}#縮放范圍
maxReplicas:6
metrics:
-type:Resource
resource:
name:cpu#指定資源指標(biāo)
target:
type:Utilization
averageUtilization:50

Prometheus集成

業(yè)務(wù)層面

項(xiàng)目依賴(lài) pom.xml

 

org.springframework.boot
spring-boot-starter-actuator


io.micrometer
micrometer-registry-prometheus

定義訪問(wèn)端口、路徑及權(quán)限 application.yaml

management:
server:
port:50000#啟用獨(dú)立運(yùn)維端口
metrics:
tags:
application:${spring.application.name}
endpoints:
web:
exposure:
base-path:/actuator#指定上下文路徑，啟用相應(yīng)端點(diǎn)
include:metrics,prometheus

將暴露/actuator/metric和/actuator/prometheus接口，訪問(wèn)方式如下：

http://127.0.0.1:50000/actuator/metric
http://127.0.0.1:50000/actuator/prometheus

運(yùn)維層面

deployment.yaml

apiVersion:apps/v1
kind:Deployment
spec:
template:
metadata:
annotations:
prometheus:io/port:"50000"
prometheus.io/path:/actuator/prometheus#在流水線中賦值
prometheus.io/scrape:"true"#基于pod的服務(wù)發(fā)現(xiàn)

配置分離

方案：通過(guò)configmap掛載外部配置文件，并指定激活環(huán)境運(yùn)行

作用：配置分離，避免敏感信息泄露；鏡像復(fù)用，提高交付效率

通過(guò)文件生成configmap

#通過(guò)dry-run的方式生成yaml文件
kubectlcreatecm-n--from-file=application-test.yaml--dry-run=1-oyaml>configmap.yaml

#更新
kubectlapply-fconfigmap.yaml

掛載configmap并指定激活環(huán)境

apiVersion:apps/v1
kind:Deployment
metadata:
name:{APP_NAME}
labels:
app:{APP_NAME}
spec:
template:
spec:
containers:
-name:{APP_NAME}
image:{IMAGE_URL}
imagePullPolicy:Always
env:
-name:SPRING_PROFILES_ACTIVE#指定激活環(huán)境
value:test
volumeMounts:#掛載configmap
-name:conf
mountPath:"/app/config"#與Dockerfile中工作目錄一致
readOnly:true
volumes:
-name:conf
configMap:
name:{APP_NAME}

匯總配置

業(yè)務(wù)層面

項(xiàng)目依賴(lài) pom.xml

 

org.springframework.boot
spring-boot-starter-actuator


io.micrometer
micrometer-registry-prometheus

定義訪問(wèn)端口、路徑及權(quán)限 application.yaml

spring:
application:
name:project-sample
profiles:
active:@profileActive@
lifecycle:
timeout-per-shutdown-phase:30s#停機(jī)過(guò)程超時(shí)時(shí)長(zhǎng)設(shè)置30s，超過(guò)30s，直接停機(jī)

server:
port:8080
shutdown:graceful#默認(rèn)為IMMEDIATE，表示立即關(guān)機(jī)；GRACEFUL表示優(yōu)雅關(guān)機(jī)

management:
server:
port:50000#啟用獨(dú)立運(yùn)維端口
metrics:
tags:
application:${spring.application.name}
endpoint:#開(kāi)啟shutdown和health端點(diǎn)
shutdown:
enabled:true
health:
probes:
enabled:true
endpoints:
web:
exposure:
base-path:/actuator#指定上下文路徑，啟用相應(yīng)端點(diǎn)
include:health,shutdown,metrics,prometheus

運(yùn)維層面

確保dockerfile模版集成curl工具，否則無(wú)法使用curl命令

FROMopenjdk:8-jdk-alpine
#構(gòu)建參數(shù)
ARGJAR_FILE
ARGWORK_PATH="/app"
ARGEXPOSE_PORT=8080

#環(huán)境變量
ENVJAVA_OPTS=""
JAR_FILE=${JAR_FILE}

#設(shè)置時(shí)區(qū)
RUNln-sf/usr/share/zoneinfo/Asia/Shanghai/etc/localtime&&echo'Asia/Shanghai'>/etc/timezone
RUNsed-i's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g'/etc/apk/repositories
&&apkadd--no-cachecurl
#將maven目錄的jar包拷貝到docker中，并命名為for_docker.jar
COPYtarget/$JAR_FILE$WORK_PATH/


#設(shè)置工作目錄
WORKDIR$WORK_PATH


#指定于外界交互的端口
EXPOSE$EXPOSE_PORT
#配置容器，使其可執(zhí)行化
ENTRYPOINTexecjava$JAVA_OPTS-jar$JAR_FILE

k8s部署模版deployment.yaml

apiVersion:apps/v1
kind:Deployment
metadata:
name:{APP_NAME}
labels:
app:{APP_NAME}
spec:
selector:
matchLabels:
app:{APP_NAME}
replicas:{REPLICAS}#Pod副本數(shù)
strategy:
type:RollingUpdate#滾動(dòng)更新策略
rollingUpdate:
maxSurge:1
maxUnavailable:0
template:
metadata:
name:{APP_NAME}
labels:
app:{APP_NAME}
annotations:
timestamp:{TIMESTAMP}
prometheus.io/port:"50000"#不能動(dòng)態(tài)賦值
prometheus.io/path:/actuator/prometheus
prometheus.io/scrape:"true"#基于pod的服務(wù)發(fā)現(xiàn)
spec:
affinity:#設(shè)置調(diào)度策略，采取多主機(jī)/多可用區(qū)部署
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
-weight:100
podAffinityTerm:
labelSelector:
matchExpressions:
-key:app
operator:In
values:
-{APP_NAME}
topologyKey:"kubernetes.io/hostname"#多可用區(qū)為"topology.kubernetes.io/zone"
terminationGracePeriodSeconds:30#優(yōu)雅終止寬限期
containers:
-name:{APP_NAME}
image:{IMAGE_URL}
imagePullPolicy:Always
ports:
-containerPort:{APP_PORT}
-name:management-port
containerPort:50000#應(yīng)用管理端口
readinessProbe:#就緒探針
httpGet:
path:/actuator/health/readiness
port:management-port
initialDelaySeconds:30#延遲加載時(shí)間
periodSeconds:10#重試時(shí)間間隔
timeoutSeconds:1#超時(shí)時(shí)間設(shè)置
successThreshold:1#健康閾值
failureThreshold:9#不健康閾值
livenessProbe:#存活探針
httpGet:
path:/actuator/health/liveness
port:management-port
initialDelaySeconds:30#延遲加載時(shí)間
periodSeconds:10#重試時(shí)間間隔
timeoutSeconds:1#超時(shí)時(shí)間設(shè)置
successThreshold:1#健康閾值
failureThreshold:6#不健康閾值
resources:#容器資源管理
limits:#資源限制（監(jiān)控使用情況）
cpu:0.5
memory:1Gi
requests:#最小可用資源（靈活調(diào)度）
cpu:0.1
memory:200Mi
env:
-name:TZ
value:Asia/Shanghai
---
kind:HorizontalPodAutoscaler#彈性伸縮控制器
apiVersion:autoscaling/v2beta2
metadata:
name:{APP_NAME}
spec:
scaleTargetRef:
apiVersion:apps/v1
kind:Deployment
name:{APP_NAME}
minReplicas:{REPLICAS}#縮放范圍
maxReplicas:6
metrics:
-type:Resource
resource:
name:cpu#指定資源指標(biāo)
target:
type:Utilization
averageUtilization:50

審核編輯：湯梓紅