k3s-server job from k3s-wrapper/0.8.1

Github source: c71a176 or master branch

Properties¶

`containerd_registry`¶

containerd registry configuration

`k3s`¶

additional-manifests¶

array of additionnel yaml to apply at k3s start
Default
[]
additional_tls_sans¶

array of additionnal tls-san for k8s api (added to default, ip, bosh-dns alias, or master vip if set)
Default
[]
api-server-tracing-config-file¶

if set, defines and activate k8s api tracing config. See https://kubernetes.io/docs/concepts/cluster-administration/system-traces/#kube-apiserver-traces

audit-policy-file¶

if set, defines and activate audit policy for k8s server
Default
  |+
    apiVersion: audit.k8s.io/v1 # This is required.
    kind: Policy
  
    # Prevent requests in the RequestReceived stage from generating audit events.
    omitStages:
      - "RequestReceived"
  
    rules:
      # Log "pods/log", "pods/status" at Metadata level
      - level: Metadata
        resources:
        - group: ""
          resources: ["pods/log", "pods/status"]
      # Exclude logging requests to a configmap called "controller-config"
      - level: None
        resources:
        - group: ""
          resources: ["configmaps"]
          resourceNames: ["controller-config"]
      # Don't log watch requests by the "system:kube-proxy" on endpoints or services
      - level: None
        users: ["system:kube-proxy"]
        verbs: ["watch"]
        resources:
        - group: "" # core API group
          resources: ["endpoints", "services"]
      # Log deployment changes at RequestResponse level
      - level: Metadata
        resources:
        - group: ""
          resources: ["deployments"]
      # Log service changes at metadata level
      - level: Metadata
        resources:
        - group: ""
          resources: ["services"]
      # Log the request body of configmap changes in the kube-system namespace.
      - level: Request
        resources:
        - group: "" # core API group
          resources: ["configmaps"]
        # You can use an empty string [""] to select resources not associated with a namespace.
        namespaces: ["kube-system"]
      # Log configmap and secret changes in all other namespaces at the Metadata level.
      - level: Metadata
        resources:
        - group: "" # core API group
          resources: ["secrets", "configmaps","serviceaccounts/token"] #see https://github.com/kubernetes/kubernetes/issues/98612#issuecomment-962088315
      # Log all other resources in core and extensions at the Request level.
      - level: Request
        resources:
        - group: "" # core API group
        - group: "extensions" # Version of group should NOT be included.
      # A wild-card rule to log all other requests at the Metadata level.
      - level: Metadata
        # Long-running requests like watches that fall under this rule will not
        # generate an audit event in RequestReceived.
        omitStages:
          - "RequestReceived"
bind-address value¶

(listener) k3s bind address (default: 0.0.0.0)
Default
0.0.0.0
bosh-post-start-delay-seconds¶

bosh post start tempo, to let the kubelet start the pods before bosh triggers another node update
Default
30
cluster-cidr¶

(networking) IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16)

cluster-dns¶

(networking) IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10)

containerd_additional_env_vars¶

additional env vars (name/value map array) to set for containerd (the key will be prefixed with CONTAINERD_, and set in k3s launch context
Default
[]
datastore-cafile¶

TLS Certificate Authority file used to secure datastore backend communication

datastore-certfile¶

TLS certification file used to secure datastore backend communication

datastore-endpoint¶

Specify etcd, Mysql, Postgres, or Sqlite (default) data source name

datastore-keyfile¶

TLS key file used to secure datastore backend communication

disable¶

(components) Do not deploy packaged components and delete any deployed components (valid itemms are coredns, servicelb, traefik, local-storage, metrics-server)
Default
[]
disable-cloud-controller¶

(components) If set, Disable k3s default cloud controller manager

disable-kube-proxy¶

(components) Disable running kube-proxy

disable-network-policy¶

(components) Disable k3s default network policy controller

disable-vxlan-hardware-options¶

Disable VxLAN harware options on private interface
Default
  - tx-udp_tnl-segmentation
  - tx-udp_tnl-csum-segmentation
do-not-killall-on-post-stop¶

if set, the bosh post-stop script wont leverage k3s-killall.sh script
Default
false
drain¶
delete-emptydir-data¶

continue even if there are pods using emptyDir (local data that will be deleted when the node is drained).
Default
true
disable-eviction¶

force drain to use delete, even if eviction is supported. This will bypass checking PodDisruptionBudgets, use with caution
Default
false
grace-period¶

period of time in seconds given to each pod to terminate gracefully. If negative, the default value specified in the pod will be used.
Default
-1
ignore-daemonsets¶

Ignore DaemonSet-managed pods.
Default
true
pod-selector¶

Label selector to filter pods on the node

selector¶

Selector (label query) to filter on

skip-wait-for-delete-timeout¶

If pod DeletionTimestamp older than N seconds, skip waiting for the pod. Seconds must be greater than 0 to skip.
Default
0
timeout¶

The length of time to wait before giving up, zero means infinite
Default
90s
embedded-ha-etcd¶

if set, use embedded etcd in ha mode. requires an odd number of servers, overrides datastore-endpoints

flannel-backend¶

(networking) One of ‘none’, ‘vxlan’, ‘ipsec’, or ‘wireguard’
Default
vxlan
kube-apiserver-arg¶

(flags) Customized flag for kube-apiserver process
Default
[]
kube-cloud-controller-manager-arg¶

(flags) Customized flag for kube-cloud-controller-manager process
Default
[]
kube-controller-manager-arg¶

(flags) Customized flag for kube-controller-manager process
Default
[]
kube-proxy-arg¶

(agent/flags) Customized flag for kube-proxy process
Default
[]
kube-scheduler-arg¶

(flags) Customized flag for kube-scheduler process
Default
[]
kubelet-args¶

(agent/flags) Customized flag for kubelet process
Default
[]
kubelet-config-file¶

content of kubelet config file, to enable eg GraceFull Node Shutdown
Default
  |+
    apiVersion: kubelet.config.k8s.io/v1beta1
    kind: KubeletConfiguration
    shutdownGracePeriod: 30s
    shutdownGracePeriodCriticalPods: 10s
master_vip_api¶

externaly defined vip ip for HA k3s (enables multi master instance groups). This is used for public api access, tls-san, and agents to server communication

node-labels¶

(agent/node) Registering and starting kubelet with set of labels
Default
[]
node-taints¶

(agent/node) Registering kubelet with set of taints.format is key=value:Effect)
Default
[]
node_name_prefix¶

explicitly set k8s node name. If not set, - is set automatically. If set, name is -

service-cidr¶

(networking) IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16)

set-provider-id-prefix¶

If set, the default provider id (k3s://- will be set as kubelet arg as ://-

token¶

(cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN]

token-file-content¶

token-file content. see https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file

v¶

(logging) Number for the log level verbosity (default: 0)
Default
0
vmodule¶

Comma-separated list of FILE_PATTERN=LOG_LEVEL settings for file-filtered logging

`registry`¶

mirrors¶

tls¶

ca¶

private registry ca

cert¶

private registry certificate

key¶

private registry private key

Templates¶

Templates are rendered and placed onto corresponding instances during the deployment process. This job's templates will be placed into /var/vcap/jobs/k3s-server/ directory (learn more).

bin/ctl (from bin/ctl.erb)
bin/drain (from bin/drain.erb)
bin/envrc (from bin/envrc)
bin/k3s-killall.sh (from bin/k3s-killall.sh)
bin/post-deploy (from bin/post-deploy.erb)
bin/post-start (from bin/post-start.erb)
bin/post-stop (from bin/post-stop.erb)
bin/pre-start (from bin/pre-start.erb)
bin/pre-stop (from bin/pre-stop.erb)
bin/setup-user-env (from bin/setup-user-env.erb)
config/additional-manifest.yaml (from config/additional-manifest.yaml.erb)
config/api-server-tracing-config.yaml (from config/api-server-tracing-config.yaml.erb)
config/audit-policy.yaml (from config/audit-policy.yaml.erb)
config/bpm.yml (from config/bpm.yml)
config/datastore-cafile (from config/datastore-cafile.erb)
config/datastore-certfile (from config/datastore-certfile.erb)
config/datastore-keyfile (from config/datastore-keyfile.erb)
config/kubelet-config.yaml (from config/kubelet-config.yaml.erb)
config/registries.yaml (from config/registries.yaml.erb)
config/registry.ca (from config/registry.ca.erb)
config/registry.cert (from config/registry.cert.erb)
config/registry.key (from config/registry.key.erb)
config/token.csv (from config/token.csv.erb)

Packages¶

Packages are compiled and placed onto corresponding instances during the deployment process. Packages will be placed into /var/vcap/packages/ directory.

This job relies on no runtime packages.

k3s-server job from k3s-wrapper/0.8.1

Properties¶

containerd_registry¶

k3s¶

additional-manifests¶

additional_tls_sans¶

api-server-tracing-config-file¶

audit-policy-file¶

bind-address value¶

bosh-post-start-delay-seconds¶

cluster-cidr¶

cluster-dns¶

containerd_additional_env_vars¶

datastore-cafile¶

datastore-certfile¶

datastore-endpoint¶

datastore-keyfile¶

disable¶

disable-cloud-controller¶

disable-kube-proxy¶

disable-network-policy¶

disable-vxlan-hardware-options¶

do-not-killall-on-post-stop¶

drain¶

delete-emptydir-data¶

disable-eviction¶

grace-period¶

ignore-daemonsets¶

pod-selector¶

selector¶

skip-wait-for-delete-timeout¶

timeout¶

embedded-ha-etcd¶

flannel-backend¶

kube-apiserver-arg¶

kube-cloud-controller-manager-arg¶

kube-controller-manager-arg¶

kube-proxy-arg¶

kube-scheduler-arg¶

kubelet-args¶

kubelet-config-file¶

master_vip_api¶

node-labels¶

node-taints¶

node_name_prefix¶

service-cidr¶

set-provider-id-prefix¶

token¶

token-file-content¶

v¶

vmodule¶

registry¶

mirrors¶

tls¶

ca¶

cert¶

key¶

Templates¶

Packages¶

`containerd_registry`¶

`k3s`¶

`additional-manifests`¶

`additional_tls_sans`¶

`api-server-tracing-config-file`¶

`audit-policy-file`¶

`bind-address value`¶

`bosh-post-start-delay-seconds`¶

`cluster-cidr`¶

`cluster-dns`¶

`containerd_additional_env_vars`¶

`datastore-cafile`¶

`datastore-certfile`¶

`datastore-endpoint`¶

`datastore-keyfile`¶

`disable`¶

`disable-cloud-controller`¶

`disable-kube-proxy`¶

`disable-network-policy`¶

`disable-vxlan-hardware-options`¶

`do-not-killall-on-post-stop`¶

`drain`¶

`delete-emptydir-data`¶

`disable-eviction`¶

`grace-period`¶

`ignore-daemonsets`¶

`pod-selector`¶

`selector`¶

`skip-wait-for-delete-timeout`¶

`timeout`¶

`embedded-ha-etcd`¶

`flannel-backend`¶

`kube-apiserver-arg`¶

`kube-cloud-controller-manager-arg`¶

`kube-controller-manager-arg`¶

`kube-proxy-arg`¶

`kube-scheduler-arg`¶

`kubelet-args`¶

`kubelet-config-file`¶

`master_vip_api`¶

`node-labels`¶

`node-taints`¶

`node_name_prefix`¶

`service-cidr`¶

`set-provider-id-prefix`¶

`token`¶

`token-file-content`¶

`v`¶

`vmodule`¶

`registry`¶

`mirrors`¶

`tls`¶

`ca`¶

`cert`¶

`key`¶