New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS: We should consider updating/reusing cluster-autoscaler to support AWS #11935
Comments
A group of us have been discussing node autoscaling this week, including @bgrant0607 @vmarmol @davidopp @jszczepkowski @piosz @gmarek @mwielgus @wojtek-t (probably forgetting some people) |
One thing we talked about was maybe layering the system like this:
|
That makes a lot of sense to me. I would love to be involved in any discussions. AWS auto-scaling-groups (and I believe Google MIGs via autoscalers) allow for a quick-and-dirty version of this. Your approach is infinitely better, though I suspect will take a little longer! The fact that the scheduler will avoid overloading the cluster makes auto-scaling externally much less useful, so we would be in custom metric territory. Even then, I think that having the master node manage the instances will be a much better experience. Maybe we could promote this interface out of pkg/cloudprovider/aws (currently used only for e2e tests): |
Hi everyone, I too am very interested in seeing progress on this front. I would really appreciate it if someone could point me to any resources / pull requests that have been done. Thanks! |
+1 |
2 similar comments
+1 |
+1 |
If anyone from would like to integrate Kubernetes with AWS autoscaler I'm happy to share our experience with integrating Kubernetes with GCE autoscaler. |
@piosz I would be interested in hearing your experience with Kubernetes GCE autoscaler and I may be interested in helping with this feature. |
@piosz I would be interested on hearing about your experience and helping with this feature as well. |
@piosz Yes, please! Very interested in this. |
Any schedules about this feature? Or is there any more detailed discussion about this? |
For 1.3 we have a plan to revisit cluster autoscaling in Kubernetes and make it more user-friendly. At this moment we are discussing our 1.3 priorities and project assignments internally at Google. We will let you once we reach some agreement regarding the possible scope of the improvement that can be delivered by Google and integration plans for other cloud providers (we will definitely need community help there). |
Our current AWS scaling strategy for Kubernetes currently has 3 parts
|
@pbitty Have you made any progress on this issue? I'd like to help anyway i can. |
#1377 might be what you're looking for On Tue, Aug 23, 2016 at 10:14 PM Abhinav Pobbati notifications@github.com
|
@philk I think the mentioned PR is not what we are looking for. Is there a typo or similar? |
kubernetes-retired/contrib#1311 is probably the link you are looking for. It references other PRs that have been opened regarding cluster autoscaler for AWS |
Oh, yeah I was on mobile and didn't realize which repo I was in kubernetes-retired/contrib#1377 was what I meant. (Though 1311 above is useful too) |
What's the status on this feature? I came across this blog: http://blog.kubernetes.io/2016/07/autoscaling-in-kubernetes.html which said AWS auto scaling would be coming in 1.3. The current stable version is 1.3.6, but I can't find any info on this. The AWS getting started doc says the max / desired instances in the AWS auto scaling group can be set, but do the new AWS instances auto register themselves? |
This blogpost was released after 1.3 and said that AWS support will be ready soon. AFAIK it's already the case. @mwielgus Can you please verify? Are there instructions how to set it up? Have we released the image? Does it require kubernetes 1.4 or is it just starting different add-on? |
There's a README here. I don't think an official image was made so you would have to fork the contrib repo and build/push the image yourself for now. |
New to the thread, trying to set up auto-scaling with k8s in aws. Is this supported now? I check the README, but not sure exactly what needs to be done to build/push the image. Any update would be really appreciated. Thanks. |
@btdlin you have to build your own docker image on whatever revision we started supporting cluster autoscaler on AWS and push it to your own registry. If you don't want to do that my company has published a public image for our own use cases which has AWS support on it |
Is there going to be an official image for the autoscaler? Why make people build it for themselves? |
Thanks @andrewsykim . Looks like v1.4 just released a few hours ago, do we know if autoscaler in aws is included in v1.4? |
@jimmycuadra yes I believe there will be an official docker image already, we just didn't know if it the published one supported aws as a cloud provider so we built our own. |
@mwielgus Can we make sure that cluster autoscaler image is released to an official repo? And I think we should close this issue now, as we support AWS :) |
@fgrzadkowski Hi, I am very much interested in this feature but I find the lack of documentation to be an issue. The README leaves a bit to be desired.
I am a heavy AWS user but new to Kubernetes if that helps understand the audience. |
Once there is an official image for it, let's make sure the docs for the autoscaler mention where it is! |
We already have a PR inflight for better documentation - kubernetes-retired/contrib#1731 @mwielgus I think that to improve documentation we will also need:
@mwielgus Can we close this issue as fixed? |
@erictune @sstarcher Does monitor the These two type events will be emit when pods can not be scheduled when the required resource(CPU/Memory) can not be fulfilled. |
Quick comments - events where not designed to be the API that other components should depend on. That's why we added |
@fgrzadkowski IIUC, you mean that event is not reliable and event is not designed to be depended on for usage like this. The most reliable way is to querying pod info and checking for After reading the source code, it seems that it will emit a |
Scheduler will emit events, but they are not considered to be part of the api for other components. Yes, you should just check pod condition, which is part of PodStatus. |
Hey team, Is there any k8s version aimed for the this feature to be released? Any suggestions for implementing this with AWS ASG + custom metrics? |
The current version of Cluster Autoscaler (0.4.0) supports AWS ASG. Closing the issue. |
BTW, Cluster Autoscaler is not driven by metrics but rather by the real need for a new node because some pods cannot schedule. |
We have an autoscaling group for the minions; we should consider enabling auto-scaling based e.g. on CPU or a custom metric we publish.
The text was updated successfully, but these errors were encountered: