Short Names and Categories

Like native resources, custom resources might have long resources names. CRs can have short names as well.

Again, kubectl learns about short names via discovery information.

1apiVersion: apiextensions.k8s.io/v1beta1
2kind: CustomResourcesDefinition
3metadata:
4	name: ats.cnat.programming-kubernetes.info
5spec:
6	...
7	shortNames:
8	- at

Further, CRs–as any other resources–can be part of categories. The most common use is the all category, as in kubectl get all. Is lists all user-facing resources in a cluster, like pods and services.

The CRs define in the cluster can join a category or create their own category via the categories field.

1apiVersion: apiextensions.k8s.io/v1beta1
2kind: CustomResourcesDefinition
3metadata:
4	name: ats.cnat.programming-kubernetes.info
5spec:
6	...
7	categories:
8	- all

Printer Cloumns

The kubectl CLI tool users server-side printing to render the output of kubectl get. This means that it queries the API server for the columns to display and the values in each row.

Custom resources support server-side printer columns as well, via additionalPrinterColumns

 1apiVersion: apiextensions.k8s.io/v1beta1
 2kind: CustomResourcesDefinition
 3metadata:
 4	name: ats.cnat.programming-kubernetes.info
 5spec:
 6	additionalPrinterColumns: # optional
 7	- name: kubectl column name
 8	  type: OpenAPI type for the colum
 9	  format: OpenAPI format for the column # optional
10	  description: human-readdable description of the column #optional
11	  priority: integer, always zero supported by kubectl
12	  JSONPath: JSON path inside the CR for the despaly value

With this, the example CRD from the introduction cloud be extend with addtionalPrinterColumns like this:

 1additionalPrinterColumns: # optional
 2- name: schedule
 3  type: string
 4  JSONPath: .spec.schedule
 5- name: command
 6  type: string
 7  JSONPath: .spec.command
 8- name: phase
 9  type: string
10  JSONPath: .status.phase

Then the kubectl would render a cnat resource as follows:

1$ kubectl get ats
2Name  SCHEDULE                    COMMAND               PHASE
3foo   2019-07-03T02:00:00Z     echo "hello world"       Pending

A Developer’s View on Custom Resource

Dynamic Client

Typed Client

Anatomy of a type

Kinds are represented as Golang structs. Usually the struct is named as the kind and is placed in a package corresponding to the group and version of the GVK at hand. A common convention is to place the GVK group/version.Kind into a Go package pkg/apis/group/version and Define a Golang struct Kind int the file types.go.

Every Golang type corresponding to a GVK embeds the TypeMeta struct from the package k8s.io/apimachinery/pkg/apis/meta/v1. TypeMeta just contains of the Kind and ApiVersion fields.

1type TypeMeta struct{
2    APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
3    Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
4}

In addition, every too-level kind–that is, one that has its own endpoint and therefore one(or more) corresponding GVRs–has to store a name, a namespace for namespaced resources, add a pretty long number of further metelevel fields. All these are stored in a struct called ObjectMeta in the package k8s.io/apimachinery/pkg/meta/v1:

1type ObjectMeta struct{
2    Name string `json:"name,omitempty"`
3    Namespace string `json:"namespace,omitempty"`
4    Labels map[string]string 
5    Annotation map[string]string
6}

Kubernetes top-level types look very similar to each other in the sense that they usually have a spec and a status. See this example of a deployment from k8s.io/kubernetes/apps/v1/types.go

1type Deployment struct{
2    metav1.TypeMeta `json:",inline"`
3    metav1.ObjectMeta `json:"metadata,omitempty"`
4    
5    Spec DeploymentSpec `json:"spec,ommitempty"`
6    Status DeploymentStatus `json:"status,omitempty"`
7}

While actual content of the types for spec and status differs significantly between different types, this split into spec and status ia a common theme or event a convention in Kubernetes, though it’s not technically required. Hence, it is good practice to follow this sturcture of CRDs as well.

Golang package structure

As we have seen, the Golang types are traditionally placed in a file called types.go in the package pkg/apis/group/version. In addition to that file, there are couple more files we want to go through now. Some of them are manually written by the developer, while some are genereated with code generators.

The doc.go file describes the API’s purpose and includes a number of package-global code generation tags:

1// package v1alpha1 contains the xxx v1alpha1 API Group
2
3// +k8s:deepcopy-gen=package
4// +k8s:protobuf-gen=package
5// +k8s:openapi-gen=true
6// +k8s:prerelease-lifecycle-gen=true
7// +groupName=cnat.programming-kubernetes.info
8package v1alpha1

Next, register.go includes helpers to register the custom resources Golang types into a scheme.

 1package version
 2import (
 3	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 4    "k8s.io/apimachinery/pkg/runtime"
 5    "k8s.io/apimachinery/pkg/runtime/schema"
 6    
 7    group "repo/pkg/apis/group"
 8)
 9
10// SchemeGroupVersion is group version used to register these objects
11var SchemeGroupVersion=scheme.GroupVersion{
12    Group: group.GroupName 
13    Version: "version"
14}
15
16// Kind takes an unqualified kind and retruns back a Group qualified GroupKind
17func Kind(kind string) scheme.GroupKind{
18    return schemeGroupVersion.WithKind(kind).GroupKind
19}
20// Resource takes an unqualified resource and returns a Group qualified GroupResource
21func Resource(resource string) scheme.GroupResource{
22    return schemeGroupVersion.WithResource(resource).GroupResource
23}
24
25var (
26    SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
27    AddToScheme = SchemeBuilder.AddToScheme
28)
29
30// Adds the list of known types to Scheme
31func addKnownTypes(scheme *runtime.Scheme) error{
32    scheme.AddKnownTypes(
33    	SchemeGroupVersion,
34        &SomeKind{},
35        &SomeKindList{}
36    )
37    metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
38    return nil
39}

Then, zz_generated.deepcopy.godefines deep-copy methods on the custom resource Golang top-level types. In addition, all substruct become deep-copyable as well.

Because the example use the tag +k8s:deepcopy-gen=package in doc.go, the deepcopy generation is on an opt-out basis; that is, DeepCopy methods are genereaed for every type in the package that does not opt out with +k8s:deepcopy-gen=false.

Typed Client created via client-gen

With the API package pkg/apis/group/version in place, the client generator client-gen creates a typed client, in pkg/generated/clientset/versioned by default. More precisely, the generated top-level object is a client set. It subsumes a number of API groups, versions, and resources.

The top-level files looks like the following:

Controller-runtime Client of Operator SDK and KubeBuilder

The controller-runtime project provides the basis for the opetator solutions Operator SDK adn Kubebuilder.

It uses discovery information from the API server to map the kinds to HTTP path.

Here is a quick example of how to use controller-runtime.

 1import (
 2	"flag"
 3    
 4    corev1 "k8s.io/api/core/v1"
 5    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 6    "k8s.io/client-go/kubernetes/scheme"
 7    "k8s.io/client-go/tools/clientcmd"
 8    
 9    runtimeclient "sig.k8s.io/controller-runtime/pkg/client"
10)
11
12func main(){
13    kubeconfig = flag.String("kubecofig","~/.kube/config","kubeconfig file path")
14    flag.Parse()
15    config,err := clientcmd.BuildConfigFromFlag("",*kubeconfig)
16    cl,_ := runtimeclient.New(config, clientOptions{
17        Scheme: sxcheme.Scheme
18    })
19    podList := &corev1.PodList{}
20    err:= cl.List(context.TODO(), client.InNamespace("default"), podList)
21}

The