DAG Workflow with Workflow resource #18827

sdminonne · 2015-12-17T14:28:36Z

@erictune, @soltysh As discussed here. Here it is the workflow proposal with the Workflow resource. It should improve composability and scheduling. It's heavily based on @derekwaynecarr Initializers #17305 (for dependency resource).
@bgrant0607, @mikedanese
@EricMountain-1A, @pmorie if you've some bandwidth

k8s-github-robot · 2015-12-17T14:34:36Z

Labelling this PR as size/L

k8s-bot · 2015-12-17T14:57:31Z

GCE e2e test build/test passed for commit b6921209df7d370cc4102f3568fd2eb0d79bdd05.

k8s-github-robot · 2015-12-17T15:08:41Z

The author of this PR is not in the whitelist for merge, can one of the admins add the 'ok-to-merge' label?

erictune · 2015-12-18T19:31:34Z

Note to myself: when I review this I need to first review the comments on #17787

erictune · 2015-12-19T06:44:35Z

The ExistenceDependencies of #17305 would check that an object exists. But for a workflow, you need to check that it has also reach a certain state, such as a job being complete.

erictune · 2015-12-19T06:49:56Z

docs/proposals/workflow.md

+	// If this value is nil, the default grace period will be used instead.
+	// Set this value longer than the expected cleanup time for your workflow.
+    // If downstream resources (job, pod, etc.) define their TerminationGracePeriodSeconds
+    // the biggest is taken.


What does "the biggest is taken" mean? Are you saying this value is defaulted from the max value of TerminationGracePeriodSeconds of all contained objects?

About ExistenceDependencies (more generally about ObjectDependencies). Sure #17305 is about existence but the specific behavior is going to be customized via the controller ControllerRef (if I'm not wrong).

I would simply modify field name in ObjectDependencies

type ObjectDependencies { ... DependenciesRef []ObjectReference `json:"dependenciesRef,omitempty"` // modified ControllerRef *ObjectReference `json:"controllerRef,omitempty"` ... }

and the ControllerRef is going to be assigned to a controller that check termination of the resources in Dependencies.

About "the biggest is taken", yes, I'm going to rephrase it.

sdminonne · 2015-12-19T11:18:44Z

@erictune thanks for the review, I'm waiting for other feedbacks (if any) and then I'll modify this according to your comments.

soltysh · 2015-12-21T09:29:05Z

I did quick review, I'm planning to go through it once again later today. Sorry @sdminonne it's taking this long, my hands are pretty full already :/

sdminonne · 2015-12-21T09:48:38Z

@soltysh no problem :) thanks for the review.

soltysh · 2015-12-21T16:16:34Z

docs/proposals/workflow.md

+* As a user I want to be able to define a workflow.
+* As a user I want to compose workflows.
+* As a user I want to delete a workflow (eventually cascading to running _tasks_).
+* As a user I want to debug a workflow (ability to track failure).


How about more concrete examples such as: I want to create a Job B that should depend on Job A, etc. instead of giving these high level ones?

soltysh · 2015-12-21T16:43:02Z

One more question, how one checks current state of a workflow, iow. which deps have been met already and which not?

soltysh · 2016-01-13T10:57:56Z

I'm missing in the proposal what happens when modification of a Workflow happens after it started, we've had a discussion on the topic on IRC. It would be beneficial for the proposal.

soltysh · 2016-01-13T10:59:36Z

Are you planning to extend kubectl in any way to simplify workflow management - if so please put it in the proposal as well. If none - state it as well.

erictune · 2016-01-15T16:23:51Z

I assume you plan to implement a Workflow Controller in e.g. pkg/controller/workflow. But I don't see the word controller anywhere in the doc. If you intend there to be a controller, I suggest you have a section about it, including some pseudo-code explaining what the controller does. If you don't intend such a controller, then please say so.

erictune · 2016-01-15T16:38:20Z

docs/proposals/workflow.md

+At the time of writing, to defer execution there are two discussions in the community:
+[#17305](https://github.com/kubernetes/kubernetes/pull/17305): an
+_initializer_ is a dynamically registered object which permits to select a custom controller
+to be applied to a resource. The controller verifies the dependencies.


I cannot tell if you are just pointing out the similarity to #17305 or if you are saying that Workflow will be implemented entirely via #17305, or that it will partly use #17305.

The WorkflowSpec has a nested JobSpec. Creating a nested object is not the same as creating a bare object. I don't see anything in #17305 about handling nested objects. Therefore, at a minimum, I think that there would need to be a Workflow Controller that extracts the nested objects and creates them as top-level objects. You could either have the Workflow Controller create them all immediately, and then use #17305 to sequence their start. Or you could just use the Workflow Controller to create each nested object (step) only when it is ready. I had assumed that you would do the latter, but rereading this section I'm not not sure what you intend to do.

@erictune: I'm going to supply will contain the description of the controller. The controller won't use #17305 (even if mechanism could be similar, I argue), at the beginning I thought I could re-use the same mechanism or at least share some code. Unluckily I won't be able to work on both and apparently #17305 is taking longer. I'll remove any reference to #17305.

Okay. Sounds good. Happy to revisit it when #17305 is done.

sdminonne · 2016-01-25T14:01:57Z

@erictune @soltysh thanks for the reviews and sorry for the silence on this.
I've worked on your feedback and in the next hours I'll supply a new version of the proposal.

sdminonne · 2016-01-25T14:18:43Z

@soltysh

About Workflow update and discussion we had on IRC: I'm going to add to the proposal.
About the modification for kubectl: I'll modify kubectl to include usual CRUD plus describe command: but the modification is going to be quite basic due to the limiation of an ASCII interface. Obviously workflow tools tends to have fancy UI (to represent the graph) but this will be out of the proposal (I'll clarify this point).

sdminonne · 2016-01-29T16:26:31Z

@erictune @soltysh I'm posting a new version. Several modifications:

Postponing execution paragraph is removed. In the Controller paragraph I describe how the detection of a Job or of a Workflow is detected. Reference to WIP: Initializers proposal #17305 has been removed.
Detecting run to completion paragraph is removed but more details are given discussing API and controller
Controller section is added.
Added some info Workflow updating
kubectl section is added
Code formatting removed

k8s-bot · 2016-01-29T17:02:56Z

GCE e2e test build/test passed for commit e8cf1dc.

k8s-bot · 2016-01-30T15:00:50Z

GCE e2e test build/test passed for commit e8cf1dc.

erictune · 2016-01-31T10:16:37Z

lgtm. minor comments that you can address later.

k8s-github-robot · 2016-01-31T10:29:20Z

Automatic merge from submit-queue

Auto commit by PR queue bot

sdminonne · 2016-01-31T10:50:58Z

@erictune thanks!

ravilr · 2016-10-16T19:25:54Z

@sdminonne @erictune @soltysh the proposal in this merged PR isn't found in the repo now. Is the thinking that this could be implemented outside of the core api using ThirdPartyResources? Thanks.

sdminonne · 2016-10-16T19:52:55Z

@ravilr right. I'm implementing the controller using ThirdPartyResources as an external controller.

Auto commit by PR queue bot

googlebot added the cla: yes label Dec 17, 2015

k8s-github-robot assigned brendandburns Dec 17, 2015

k8s-github-robot added kind/design Categorizes issue or PR as related to design. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 17, 2015

k8s-github-robot added the needs-ok-to-merge label Dec 17, 2015

mikedanese assigned erictune and unassigned brendandburns Dec 17, 2015

erictune reviewed Dec 19, 2015
View reviewed changes

soltysh reviewed Dec 21, 2015
View reviewed changes

erictune mentioned this pull request Dec 26, 2015

Added proposal for "acting-as". #16933

Closed

erictune mentioned this pull request Jan 13, 2016

job DAG workflow proposal #17787

Closed

sdminonne added 2 commits January 14, 2016 12:08

DAG Workflow with Workflow resource

e368636

To address review feedbacks

8b3895d

erictune reviewed Jan 15, 2016
View reviewed changes

Workflow: adding controller and address 2nd round review

e8cf1dc

sdminonne force-pushed the workflow_resource branch from 77471d2 to e8cf1dc Compare January 29, 2016 16:26

erictune added lgtm "Looks good to me", indicates that a PR is ready to be merged. e2e-not-required and removed needs-ok-to-merge labels Jan 31, 2016

k8s-github-robot pushed a commit that referenced this pull request Jan 31, 2016

Merge pull request #18827 from sdminonne/workflow_resource

a81fa2f

Auto commit by PR queue bot

k8s-github-robot merged commit a81fa2f into kubernetes:master Jan 31, 2016

sdminonne deleted the workflow_resource branch February 1, 2016 12:08

sdminonne restored the workflow_resource branch February 1, 2016 12:40

xingzhou pushed a commit to xingzhou/kubernetes that referenced this pull request Dec 15, 2016

Merge pull request kubernetes#18827 from sdminonne/workflow_resource

a7ffad5

Auto commit by PR queue bot

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAG Workflow with Workflow resource #18827

DAG Workflow with Workflow resource #18827

sdminonne commented Dec 17, 2015

k8s-github-robot commented Dec 17, 2015

k8s-bot commented Dec 17, 2015

k8s-github-robot commented Dec 17, 2015

erictune commented Dec 18, 2015

erictune commented Dec 19, 2015

erictune Dec 19, 2015

sdminonne Dec 21, 2015

sdminonne Dec 21, 2015

sdminonne commented Dec 19, 2015

soltysh commented Dec 21, 2015

sdminonne commented Dec 21, 2015

soltysh Dec 21, 2015

sdminonne Dec 22, 2015

soltysh commented Dec 21, 2015

soltysh commented Jan 13, 2016

soltysh commented Jan 13, 2016

erictune commented Jan 15, 2016

erictune Jan 15, 2016

sdminonne Jan 25, 2016

erictune Jan 25, 2016

sdminonne commented Jan 25, 2016

sdminonne commented Jan 25, 2016

sdminonne commented Jan 29, 2016

k8s-bot commented Jan 29, 2016

k8s-bot commented Jan 30, 2016

erictune commented Jan 31, 2016

k8s-github-robot commented Jan 31, 2016

sdminonne commented Jan 31, 2016

ravilr commented Oct 16, 2016

sdminonne commented Oct 16, 2016

Navigation Menu

DAG Workflow with Workflow resource #18827

DAG Workflow with Workflow resource #18827

Conversation

sdminonne commented Dec 17, 2015

k8s-github-robot commented Dec 17, 2015

k8s-bot commented Dec 17, 2015

k8s-github-robot commented Dec 17, 2015

erictune commented Dec 18, 2015

erictune commented Dec 19, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdminonne commented Dec 19, 2015

soltysh commented Dec 21, 2015

sdminonne commented Dec 21, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soltysh commented Dec 21, 2015

soltysh commented Jan 13, 2016

soltysh commented Jan 13, 2016

erictune commented Jan 15, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdminonne commented Jan 25, 2016

sdminonne commented Jan 25, 2016

sdminonne commented Jan 29, 2016

k8s-bot commented Jan 29, 2016

k8s-bot commented Jan 30, 2016

erictune commented Jan 31, 2016

k8s-github-robot commented Jan 31, 2016

sdminonne commented Jan 31, 2016

ravilr commented Oct 16, 2016

sdminonne commented Oct 16, 2016