initial template and parameterization proposal #18215
Conversation
|
Labelling this PR as size/L |
k8s-github-robot
added
kind/design
|
GCE e2e test build/test passed for commit 5da37eb6715bc34e5ac314ddd7fff51d8b33a54a. |
|
Thanks. WTAL |
|
cc @kubernetes/sig-config |
| ### Use cases for templates in general | ||
|
|
||
| * Providing a full baked application experience in a single portable object that can be repeatably deployed in different environments. | ||
| * eg Wordpress deployment with separate standalone database pod |
There was a problem hiding this comment.
Nit: a standalone pod is an anti-pattern
There was a problem hiding this comment.
you mean a pod w/o a replication controller? yeah sorry i was a bit sloppy with my language here. I was only trying to distinguish from "single pod containing both a web app an db container", not imply there wasn't also a replication controller managing the DB pod. i'll clarify in the next revision.
|
|
||
| *It should be possible to validate templates and template parameters, both values and the schema.* | ||
|
|
||
| * Template objects are subject to standard api validation. |
There was a problem hiding this comment.
Except when the value does not validate - such as replacement in names ($( is not valid)
There was a problem hiding this comment.
We should be able to validate the template field names.
As for values, if we had jsonschema-like validation specifications for parameters and/or for the resource fields, we could validate the values to some degree.
There was a problem hiding this comment.
Except when the value does not validate - such as replacement in names ($( is not valid)
we don't validate the objects within the template definition, they are just runtime.Object, right? i'm just saying the template api object is validated as such, not that the things it contains are validated as whatever type they are. that does not happen (and as you note, really can't happen until after substitution happens. This situation will be even worse once we add support for substitution into integer fields)
|
I think we need something like this. Kubernetes is about automating operations: Templates could also be used for a new generation of controllers #14961, like PetSet #18016, replacing the original idea of just pod templates #170. We also have at least half a dozen cases in flight where we need parameter substitution. |
| solutions. (tldr: a syntax like ${FOO | int} could instruct the template processor to convert the json field to an integer when doing the | ||
| substitution. This could also support secret generation by using a syntax like `${FOO | base64}` which would take the value of parameter | ||
| `$FOO` and base64 encode it before setting the json field to the resulting value) | ||
| * Another option would be to add type information to the parameter definitions, making it slightly easier to reference a parameter within a |
There was a problem hiding this comment.
Can't we use the swagger to get the type?
There was a problem hiding this comment.
only if template processing becomes schema aware which it currently is not. will add it as an option.
|
@bgrant0607 @smarterclayton addressed some of your comments, in particular changed to the I have not addressed @bgrant0607's request to support a map of parameter values + template for instantiation, i'd like to see a little more discussion on the use case for that given that we've so far not seen a need for it in openshift. blind submission of parameter key/value pairs for a template seems likely to lead to failure for a user. |
|
GCE e2e test build/test passed for commit 2d0df6401f8f8921c71e7e57130825f8ae23fc9b. |
| * Utilizes multiple files with different syntax to define an application topology. | ||
| * Does not define a first class API object. | ||
| * Does not appear to natively offer generation logic for producing parameter values. | ||
|
|
There was a problem hiding this comment.
The preceding statements about DM are inaccurate. Please read the DM documentation at https://github.com/kubernetes/deployment-manager, including the contents of the docs folder, and correct these statements.
There was a problem hiding this comment.
Utilizes multiple files with different syntax to define an application topology.
this example uses 4 files including both yaml and jinja formats: https://github.com/kubernetes/deployment-manager/tree/master/examples/wordpress
the docs also indicate templates can be written in python. none of the examples/formats match k8s api objects/syntax.
Does not define a first class API object.
see above, what DM defines/consumes is not a first class k8s api object. Perhaps there's some confusion in what i meant by api object, I did not mean that DM doesn't define an api for itself, I meant it does not define api objects that are consumed by k8s. I can clarify this statement.
Does not appear to natively offer generation logic for producing parameter values.
I understand DM has parameters, i don't see native logic for defining how a parameter value should be generated (eg generate a random password). Presumably it's possible to make that happen via jinja or python, but I wouldn't consider that native support within DM, hence my statement. Again, I can clarify this to indicate value generation is deferred to jinja/python.
If I am still not understanding the design of DM, I'm afraid I'm going to need you to be more explicit about what is inaccurate/what I have misunderstood.
There was a problem hiding this comment.
DM defines a single template usage format. It's a YAML based list of resource, each of which contains a name, type and parameters. True, the template definition can be written either in Jinja, which is simply marked up YAML, or Python, which produces YAML. A Jinja template for a k8s primitive is easily recognizable as a k8s configuration file with mustache markup.
Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.
DM can also take vanilla k8s configuration files as input. And, when a template is expanded, the result is always a flat list of vanilla k8s configuration files, which is readily accessible via the API.
DM doesn't define a k8s API object specifically because a native implementation baked into k8s has been discouraged by the preceding discussion. If that has somehow changed, we could easily make a DM template a native API object. However, we believe there are good reasons not to do that, not least of which is the well established precedent of publishing content in registries, rather than in runtimes. That said, when a configuration is deployed using DM, it is capture and stored in the cluster, providing a record of the deployment, including the parameters and expansion results.
If by native support for generation logic, you mean hard wired, as opposed to programmable, then no, DM doesn't have hard wired generators. We consider hard wired implementations to be an anti-pattern, in general.
There was a problem hiding this comment.
Also, there's no requirement to use both languages in a single example, or to use multiple files, so both assertions are inaccurate.
is it possible to write a single file, in k8s-compatible json syntax, which provides parameterization? That is what is achieved by the openshift template schema, and what I am trying to distinguish here as a desirable trait. Whether an example can be created which doesn't need additional syntax or files isn't really my point, the question is whether the full power of DM can be realized without branching out into multiple files and multiple syntaxes (above and beyond the k8s syntax).
My impression is that the answer is no. Is it a desirable goal (single file, k8s syntax, api object)? in my opinion, based on the user experience we have been able to deliver in openshift by integrating templates as a first class concept, is that it is, but I think we'll be hashing that out in the syntax/schema discussion once I update this PR per our discussion yesterday.
Regarding parameter generation, I agree with the concerns about it being compiled into the runtime and it would be nice to make it more flexible. On the other hand, by being a little opinionated here, we provided a generation function that covers probably 70% of the use cases for parameter generation (no one has come to us so far asking for additional generation functionality) in an extremely consumable format that minimizes complexity or requirements to learn new languages.
|
@bgrant0607: comments addressed in latest commit. |
|
GCE e2e test build/test passed for commit 246fd71db09197fc463a11f748710d171cf86807. |
|
LGTM. Please rebase, run hack/update-generated-docs.sh, and squash, and then I'll apply the label for merging. |
bparees
force-pushed
the
templates
branch
2 times, most recently
from
88cd5de
to
a2b5112
Compare
|
@bgrant0607 done. |
|
GCE e2e test build/test passed for commit a2b5112aefc6cdba2ef5639a8ab8bdad5d952f8d. |
|
GCE e2e test build/test passed for commit 89a6561. |
bgrant0607
added
the
lgtm
|
LGTM! |
|
Automatic merge from submit-queue |
Auto commit by PR queue bot
|
thanks @bgrant0607. Hopefully we can find some people to start the implementation come early march. |
|
Not sure if this is the best place to discuss this now that the proposal has been merged or whether or I should start a new issue, but: I've started on a CLI tool that attempts to implement client-side template processing using the format described in this proposal. You can find the program at https://github.com/InQuicker/ktmpl. It's written in Rust, but the details of the source code are not really important for discussion. There are a few issues I've coming across while working on this implementation (some minor, some more significant):
|
|
thanks for the feedback, i've pushed fixes to the syntactical issues here:
I believe there is:
there can't be such an example because the format precludes using literal text (by which I assume you mean unquoted text). That's the purpose of the $(()) syntax, to strip the quotes where appropriate.
There is an inherent assumption that the user who put the
"int" was probably overly shorthand(ie we should include other types), but the point of "base64" is again that this is intended to hint to the user of the template how the value is going to be used. Telling them the type is base64 tells them they need to provide a base64 encoded value for this parameter. The fact that ultimately that base64 value gets plugged in as a json string isn't relevant.
Required is intended to indicate that null/empty string is not an acceptable value for this parameter. The parameter must have a non-empty value either provided via the default, or provided by the user. I will update this in the above PR as well. |
|
Thank you, @bparees, that was very helpful! Your changes in that PR do well to clarify the things that tripped me up. When I said "there are no examples of the two types of interpolation combined in one value and there aren't examples of each of the types of interpolation surrounded by literal text," I meant that the example template doesn't include those forms, so you don't see them in a realistic context and you can't use that example as a very complete test case. That's a minor issue, though, and I can adjust for my own needs. Thanks again! |
|
Not sure if this is the best place to ask that the proposal has been merged, but I can not find the progress of this proposal's implement anywhere. How about this proposal? |
|
@jolestar it has not been implemented and at this point seems unlikely to be. |
|
@jolestar A client-based implementation is mentioned above: https://github.com/InQuicker/ktmpl Openshift also has an implementation. I agree with @bparees that it is unlikely to be implemented at this point. My current thinking on this and related topics is here: |
|
Thank you @bparees @bgrant0607 for reply |
@bgrant0607 @brendandburns @smarterclayton @derekwaynecarr
This is a proposal to define a syntax for authoring templates that represent k8s object topologies, while allowing those templates to be customized prior to instantiation via a parameterization mechanism.
Related discussions/template proposals:
#14918
#14993
#4210
#10408 (comment)
#11492