Home未分类如何自定义 Kubernetes CRD

如何自定义 Kubernetes CRD

2009 年 3 月 27 日


kubebuilder 安装文档可以参考官档 
Install kubebuilder
 

os=$(go env GOOS)
arch=$(go env GOARCH)


# download kubebuilder and extract it to tmp
curl -L https://go.kubebuilder.io/dl/2.3.1/${os}/${arch} | tar -xz -C /tmp/


# move to a long-term location and put it on your path
# (you'll need to set the KUBEBUILDER_ASSETS env var if you put it somewhere else)
sudo mv /tmp/kubebuilder_2.3.1_${os}_${arch} /usr/local/kubebuilder
export PATH=$PATH:/usr/local/kubebuilder/bin



 03 kubebuilder初始化 

kubebuilder 代码初始化：

# github.com/opskumu 以实际仓库为主，本实例为 crd-example
mkdir -p $GOPATH/src/github.com/opskumu/crd-example
cd $GOPATH/src/github.com/opskumu/crd-example
# 域名以实际域名为主，本实例为 opskumu.com
kubebuilder init --domain opskumu.com
kubebuilder edit --multigroup=true

查看当前生成代码的目录结构：

# tree -L 2 ./
.
├── Dockerfile
├── Makefile
├── PROJECT
├── bin
│   └── manager
├── config
│   ├── certmanager
│   ├── default
│   ├── manager
│   ├── prometheus
│   ├── rbac
│   └── webhook
├── go.mod
├── go.sum
├── hack
│   └── boilerplate.go.txt
└── main.go

 04 kubebuilder创建API 

# kubebuilder create api --group test --version v1 --kind ConfigMapHistory
Create Resource [y/n]    # 创建 ConfigMapHistory 资源，此处选择 y
y
Create Controller [y/n]  # 创建控制器，此处不需要，选择 n
n
# tree apis/test/v1   # apis 目录存放 group test v1 相关的资源 API
apis/test/v1
├── configmaphistory_types.go
├── groupversion_info.go
└── zz_generated.deepcopy.go

 05 自定义代码生成client代码 

默认生成apis/test/v1/configmaphistory_types.go代码如下: 

......
// +kubebuilder:object:root=true


// ConfigMapHistory is the Schema for the configmaphistories API
type ConfigMapHistory struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec   ConfigMapHistorySpec   `json:"spec,omitempty"`
    Status ConfigMapHistoryStatus `json:"status,omitempty"`
}
......



我们创建 ConfigMapHistory 类型的资源的目的是存放 ConfigMap 的历史记录，所以需要针对以上代码做一些修改：

......
// +genclient
// +kubebuilder:object:root=true


// ConfigMapHistory is the Schema for the configmaphistories API
type ConfigMapHistory struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`


    ConfigMap  string            `json:"configMap"`                // 关联 ConfigMap 名
    Data       map[string]string `json:"data,omitempty"`           // 对应 ConfigMap 版本数据
    BinaryData map[string]byte   `json:"binaryData,omitempty"`     // 对应 ConfigMap 版本二进制数据
}
......

注意 
// +genclient
 是额外添加的，为后续 
code-generator
 生成 client 代码打标签。types 文件修改之后，可以执行 
make manifests
 指令生成 CRD 文件：

# make manifests
# ls config/crd/bases
test.opskumu.com_configmaphistories.yaml

如果代码结构变更，需要再次执行 make manifest 以生成新的 CRD 文件创建 apis/test/v1/doc.go 、 apis/test/v1/register.go 、 hack/tools.go 和 hack/update-codegen.sh 文件

  • apis/test/v1/doc.go 
/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/


// +k8s:deepcopy-gen=package
// 注意 groupName 和实际相同
// +groupName=test.opskumu.com


// Package v1 is the v1 version of the API.
package v1

  • apis/test/v1/register.go

/*
Copyright 2017 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/


package v1


import (
    "k8s.io/apimachinery/pkg/runtime/schema"
)


// SchemeGroupVersion is group version used to register these objects.
var SchemeGroupVersion = GroupVersion


func Resource(resource string) schema.GroupResource {
    return SchemeGroupVersion.WithResource(resource).GroupResource()
}

  • hack/tools.go

// +build tools


/*
Copyright 2019 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/


// This package imports things required by build scripts, to force `go mod` to see them as dependencies
package tools


import _ "k8s.io/code-generator"

tools.go
 主要为了引入 
k8s.io/code-generator
 依赖。

  • hack/update-codegen.sh

#!/usr/bin/env bash


# Copyright 2017 The Kubernetes Authors.                                                                                       #                                                                                                                              # Licensed under the Apache License, Version 2.0 (the "License");                                                              # you may not use this file except in compliance with the License.                                                             # You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


set -o errexit
set -o nounset
set -o pipefail


SCRIPT_ROOT=$(dirname "${BASH_SOURCE[0]}")/..
CODEGEN_PKG=${CODEGEN_PKG:-$(cd "${SCRIPT_ROOT}"; ls -d -1 ./vendor/k8s.io/code-generator 2>/dev/null || echo ../code-generator)}


# generate the code with:
# --output-base    because this script should also be able to run inside the vendor dir of
#                  k8s.io/kubernetes. The output-base is needed for the generators to output into the vendor dir
#                  instead of the $GOPATH directly. For normal projects this can be dropped.
bash "${CODEGEN_PKG}"/generate-groups.sh "client,informer,lister" \
  github.com/opskumu/crd-example/generated github.com/opskumu/crd-example/apis \
  test:v1 \
  --output-base "$(dirname "${BASH_SOURCE[0]}")/../../../.." \
  --go-header-file "${SCRIPT_ROOT}"/hack/boilerplate.go.txt


# To use your own boilerplate text append:
#   --go-header-file "${SCRIPT_ROOT}"/hack/custom-boilerplate.go.txt

以上代码可以参考 sample-controller 
doc.go
/
register.go
/
tools.go
/
update-codegen.sh
 内容，本文中 CRD 是 kubebuilder 生成的，因此内容稍有不同。

创建完成以上文件之后，执行以下命令生成代码：

# 注意此处版本以下几个包要一致，否则可能出现不兼容的情况
# K8SVERSION=v0.19.6
# go get -v k8s.io/code-generator@${K8SVERSION}
# go get -v k8s.io/client-go@${K8SVERSION}
# go get -v k8s.io/apimachinery@${K8SVERSION}
# go mod vendor

把相关依赖包放入 vendor 中，以便 
hack/update-codegen.sh
 可以调用 
code-generator
 中的脚本。完成之后，执行生成代码指令：

# bash hack/update-codegen.sh
Generating clientset for test:v1 at github.com/opskumu/crd-example/generated/clientset
Generating listers for test:v1 at github.com/opskumu/crd-example/generated/listers
Generating informers for test:v1 at github.com/opskumu/crd-example/generated/informers
# crd-example tree -L 1 generated
generated
├── clientset
├── informers
└── listers

至此完成了添加 ConfigMapHistory CRD 和相关 client 代码的所有操作，如果需要生成其它 Kind 类型的重复以上操作即可（以上创建的文件不需要重复创建）。

 06 创建CRD 

kubectl create -f config/crd/bases/test.opskumu.com_configmaphistories.yaml

创建测试资源

# cat test.yaml
apiVersion: "test.opskumu.com/v1"
kind: ConfigMapHistory
metadata:
  name: test
configMap: "test"
data:
  test: test
# kubectl create -f ./test.yaml
configmaphistory.test.opskumu.com/test created
# kubectl get configmaphistory.test.opskumu.com
NAME   AGE
test   27s

完整示例代码参见 
crd-example

参考文档：

Related Posts

About The Author

peace