Kubernetes 에서 Cert-manager로 letsencrypt 인증서 발급/설정

kubernetes 에 구동하는 웹에서도 https 설정이 필요하고, 무료 인증서를 사용할 경우 letsencrypt 에서 인증서를 발급받고 웹에 적용할 수 있다. 

네이버클라우드에서 테스트 해봤고, nginx ingress 를 사용했다.  (k8s cluster 는 구성이 되어 있어야 하고, nginx ingress 를 설치하면 를 생성하면, ingress 와 매칭된 NLB는 자동 생성된다.)

전체적인 구조는 이정도 되는 것 같다. 

1. NKS 에 ingress cotroller 설치 

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.47.0/deploy/static/provider/cloud/deploy.yaml

namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created


> nginx ingress cotroller 설치 확인 
kctl2 get service -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   198.19.156.158   ingress-ngi-ingress-ngin-xxxx-xxxx-xxxxxxxxxx.kr.lb.naverncp.com   80:31535/TCP,443:31064/TCP   2m2s
ingress-nginx-controller-admission   ClusterIP      198.19.150.71    <none>                                                                   443/TCP                      2m2s

 

2. cert-manager 설치

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml

customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
namespace/cert-manager created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews created
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
role.rbac.authorization.k8s.io/cert-manager:leaderelection created
role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created

> cert-manager 설치 확인
kubectl get pods -n cert-manager
NAME                                    READY   STATUS    RESTARTS   AGE
cert-manager-cainjector-f4bccfb-p75pd   1/1     Running   0          49s
cert-manager-df8f7c899-x5wzq            1/1     Running   0          49s
cert-manager-webhook-67676487d9-b4tdl   1/1     Running   0          49s

3. cluster-issuer 생성

아래 yaml 파일에서 이메일 주소만 수정하면 된다.

# vi cluster-issuer.yaml

kpiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: 자신의 이메일 주소를 기재
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
    # An empty 'selector' means that this solver matches all domains
    - selector: {}
      http01:
        ingress:
          class: nginx

---
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: 자신의 이메일 주소를 기재
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-prod
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: nginx

cluster issuer 생성 및 생성 여부 확인

kubectl create -f cluster-issuer.yaml

kubectl get clusterissuer
NAME                  READY   AGE
letsencrypt-prod      True    25s
letsencrypt-staging   True    25s

4. 인증서를 사용할 web 및 ingress 배포

# vi deploy-web.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webweb
spec:
  replicas: 3
  selector:
    matchLabels:
      app: webweb
  template:
    metadata:
      labels:
        app: webweb
    spec:
      containers:
      - name: webweb
        image: pghmv2sm.kr.private-ncr.ntruss.com/webweb:1.0 # 컨테이너 이미지는 알아서 변경
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: regcred
      nodeSelector:
        ncloud.com/nks-nodepool: kr2-node-01


apiVersion: v1
kind: Service
metadata:
  name: webweb
spec:
  type: NodePort
  selector:
    app: webweb
  ports:
    - protocol: TCP
      port: 80

kubectl create -f deploy-web.yaml

# vi webweb-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: webweb-ingress
  annotations:
    ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: nginx
    kubernetes.io/tls-acme: "true"
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - nado.gati.ga # 인증서를 적용할 도메인으로 변경
    secretName: webweb-devbox-kr-cert # 인증서 key가 저장되는 secret
  rules:
  - host: nado.gati.ga # 인증서를 적용할 도메인으로 변경
    http:
      paths:
      - path: /
        backend:
          serviceName: webweb
          servicePort: 80

kubectl create -f webweb-ingress.yaml

지금 상태에서 certificate, certificaterequest 를 확인해보면 false, pending 상태로 보여질텐데, DNS 에 Ingress NLB 도메인에 추가해주면 바로 인증서 발급이 진행된다. 

 

5. DNS 설정 변경

webweb-ingress.yaml 에서 인증서를 발급할 도메인으로 설정한 nado.gati.ga 에 ingress-nbl 도메인을 설정해준다. 

global dns 에 cname 추가 설정

Ingress controller 설치 시 자동으로 생성된 NLB 정보

6. 브라우저 접속 확인 

nado.gati.ga 접속 시 인증서 설정이 정상적으로 되어 https 로 연결되었음을 확인할 수 있다.