Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic Kubelet Configuration Proposal #29459

Closed
wants to merge 4 commits into from

Conversation

mtaufen
Copy link
Contributor

@mtaufen mtaufen commented Jul 22, 2016

This change is Reviewable

@mtaufen mtaufen mentioned this pull request Jul 22, 2016
8 tasks
@k8s-github-robot k8s-github-robot added kind/design Categorizes issue or PR as related to design. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note-label-needed labels Jul 22, 2016
@derekwaynecarr
Copy link
Member

cc @kubernetes/sig-node

@vishh vishh assigned vishh and derekwaynecarr and unassigned brendandburns Jul 23, 2016
## Motivation

The Kubelet is currently configured via command-line flags. This is painful for a number of reasons:
- It makes it difficult to change the way Kubelets are configured in a running cluster, without tearing the cluster down.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well tearing down is not necessary. It depends on the config management system that is being used.
I'd say that it requires changing the kubelet startup configuration which is often tedious.

@davidopp
Copy link
Member

cc/ @mikedanese

@k8s-github-robot k8s-github-robot added the do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. label Jul 28, 2016

- Add a well-defined, versioned configmap object for Kubelet configuration.
- Add/Remove/Update global and per-node Kubelet config on the fly via the API server.
- Provide a REST API to get the config currently in use by a given Kubelet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not place the configuration and the source of the configuration in NodeStatus?
I imagine a workflow where users can pull a new config object from a separate git repo, and have a controller apply the new config to nodes in a controlled manner, by waiting for a set of nodes to come up with the new configuration and be healthy, before proceeding with applying the new config to other nodes.
Bad config changes can be reverted via git based workflow on top of this functionality.

@mtaufen
Copy link
Contributor Author

mtaufen commented Aug 4, 2016

Tracking an example implementation of some basic functionality here: #30090.

## Motivation

The Kubelet is currently configured via command-line flags. This is painful for a number of reasons:
- It makes it difficult to change the way Kubelets are configured in a running cluster, because it is often tedious to change the Kubelet startup configuration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without a configuration management solution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ansible or salt can do that for you. You will have problems when you update kubelet configuration in apiserver which gets reflected in a kubelet vs. re-running ansible/running salt. With ansible, it is quote easy to skip configuration tasks (skip-tags flag) once the cluster is deployed and the ansible is re-run. So the new kubelet configuration does not get touched (still, need to keep that in mind). With salt, not sure if you can do anything here unless you stop the salt, change salt (e.g. skip configuration) and start salt.

Saying, some folks can be running ansible/salt already. When you start configuring the kubelet dynamically by default, you can get into trouble.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. We should warn people about this in the documentation for the feature. With the current design, it would be pretty easy to avoid dynamically configuring the Kubelet -- just choose not to post kubelet-<node-name> configmaps in kube-system.

mtaufen added a commit to mtaufen/kubernetes that referenced this pull request Jul 10, 2017
Alpha implementation of the Dynamic Kubelet Configuration feature.
See the proposal doc in kubernetes#29459.
mtaufen added a commit to mtaufen/kubernetes that referenced this pull request Aug 8, 2017
Alpha implementation of the Dynamic Kubelet Configuration feature.
See the proposal doc in kubernetes#29459.
k8s-github-robot pushed a commit that referenced this pull request Aug 9, 2017
Automatic merge from submit-queue (batch tested with PRs 50016, 49583, 49930, 46254, 50337)

Alpha Dynamic Kubelet Configuration

Feature: kubernetes/enhancements#281

This proposal contains the alpha implementation of the Dynamic Kubelet Configuration feature proposed in ~#29459~ [community/contributors/design-proposals/dynamic-kubelet-configuration.md](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md). 

Please note:
- ~The proposal doc is not yet up to date with this implementation, there are some subtle differences and some more significant ones. I will update the proposal doc to match by tomorrow afternoon.~
- ~This obviously needs more tests. I plan to write several O(soon). Since it's alpha and feature-gated, I'm decoupling this review from the review of the tests.~ I've beefed up the unit tests, though there is still plenty of testing to be done.
- ~I'm temporarily holding off on updating the generated docs, api specs, etc, for the sake of my reviewers 😄~ these files now live in a separate commit; the first commit is the one to review.

/cc @dchen1107 @vishh @bgrant0607 @thockin @derekwaynecarr 

```release-note
Adds (alpha feature) the ability to dynamically configure Kubelets by enabling the DynamicKubeletConfig feature gate, posting a ConfigMap to the API server, and setting the spec.configSource field on Node objects. See the proposal at https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-kubelet-configuration.md for details.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge DEPRECATED. Indicates that a PR should not merge. Label can only be manually applied/removed. kind/design Categorizes issue or PR as related to design. release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet