Similar to #13358, but I think this is the more desirable way to do an external integration.

Works best when we replace *_SERVICE_HOST env with DNS name, similar to #9983

Implications:

• Does a service that wants to do this need a cluster IP?
  • Only if they load balance?
    • Why would they not load balance at the remote name instead?
    • Let's not introduce a new layer of DNS caching here?
• If we don't need a cluster IP, then this is like a headless service
  • Except that headless services always have A records - CNAME is contradictory to that
• We don't want the user to use _endpoints when we have this CNAME?
• This isn't like ClusterIP, NodePort, or LoadBalancer (probably orthogonal)

This is a more likely integration scenario than #11838.

@kubernetes/goog-cluster

@thockin @bgrant0607 feelings on this? This is pretty important to us for external service integration - if we can agree it's useful I'd like to scope out the API so we can implement something. This enables two paths for external integration - when you need to manage the endpoints, and when you don't. It's strictly simpler to do CNAME integration than to load balance, and makes referencing external services much easier.

Of course CNAMEs can't let a client verify against the aliased name, only the original, and TLSA doesn't do what I would want it to do. It would be best if a client could chase the CNAMEs, but I don't see that pattern in wide use anywhere.

If hypothetically this was another spec field like externalName, the kubelet could substitute it in service host variables for the service IP. We could do that via a DNS lookup at start time without needing to call the master to fetch the service. I do think there's a strong use case for referential identity (cluster A say "db" is "oracle-mydb.mycompany.com") without attempting to proxy. However, for TLS that becomes more problematic.

In the load balancer case, the cluster signer can sign for the service (to attest that yes, the service oracle.mynamespace.svc.cluster.local refers to oracle-mydb.mycompany.com) and it could terminate TLS at the service endpoint. externalName could be used by the endpoints controller to set endpoints periodically (similar to Brendan's pull, but derived from the service spec directly).

cc @brendandburns @ArtfulCoder

This feels like a new API: ServiceAlias or somesuch.

Is it different than a service? Can a ServiceAlias and a Service overlap
with the same name? I would expect ServiceAlias to inject env vars into
pods and be represented inside the svc DNS namespace (that's partially why
I assumed service is closer).

On Wed, Sep 16, 2015 at 2:05 PM, Brian Grant notifications@github.com
wrote:

This feels like a new API: ServiceAlias or somesuch.

—
Reply to this email directly or view it on GitHub
#13748 (comment)
.

Clayton Coleman | Lead Engineer, OpenShift

As a concrete reference, here's the design work we're doing on elements of
how services would cross namespaces and be promoted and discovered within
OpenShift, as well as how an admin can control the exposure and binding of
services from on and off platform and manage dynamic provisioning:

• https://docs.google.com/document/d/1JMGtLXAVMDCtuHf8HovaEuXzZJCBPU_x_SZrQuSekqg/edit?usp=sharing
• https://docs.google.com/presentation/d/1-KpO0C_H4pqBK5t3FPmfrGT0yo-hYHZAQhOqzGpQCwc/edit?usp=sharing

We want to have very lightweight service-like objects in order to
accomplish this. Those lightweight objects might point to external cluster
objects or internal cluster objects, but we wanted to make it easy to
describe the relationship between the local stub (or ref) and the remote
service (on or off cluster). Subdivision of clusters is concretely
happening today, so an object ref across namespaces isn't sufficient to
declare the desire to allow cluster subdivision boundaries to be breached
(it's not enough for user A to say I want service to be exposed to B, B has
to say they want it, and possibly admin C has to approve it).

@kubernetes/rh-cluster-infra @kubernetes/rh-networking

cc @ggogolowicz

@smarterclayton this got buried like so many other ideas. What's the
status of CNAME desires? I feel there's something here, but there are a
lot of gotchas.

On Mon, Nov 30, 2015 at 2:51 PM, Brian Grant notifications@github.com
wrote:

cc @ggogolowicz https://github.com/ggogolowicz

—
Reply to this email directly or view it on GitHub
#13748 (comment)
.

@thockin we are still interested in finding a way to solve the underlying use case, where there is an off-cluster service that we want to manage and treat like an on-cluster service. This meshes with the service catalog/linking proposal (#17543), which I still need to find time to do another update pass to.

I think one of the biggest hurdles is TLS hostname verification if we proceed with CNAMEs.

And I have not had any clever ideas other than making it easy for the
downward API to do the indirection, so you at least get an env var with a
name.

In practice, a lot of serious shops use ramp nodes or other intermediate
proxies for egress traffic off cluster - it's likely that a route/ingress
load balancer configured to point to the external service can do tls
rencrypt, which in theory is a nice compromise. The open question is how
you safely isolate the certificates that sign for svc.cluster.local so that
an egress router compromise does not result in the need to rekey the
cluster.

So, hypothetically:

Define a new service type / behavior "reference"
Allow a name to be specified that results in DNS CNAMEing to the ref
The ref can be an egress load balancer
Have a kube-proxy/ingress/route instance balancer set up that handles SNI
(for tls) at the ref or http otherwise.

I think the goal should be to allow that composition without enforcing it.
A non-tls service is perfectly fine with CNAME only - a tls service
requires SNI proxying with CNAME, which coincidentally we've already solved
in several places. That also enables egress ramp nodes (to coexist with
existing firewall approaches).

On Dec 1, 2015, at 9:13 AM, Andy Goldstein notifications@github.com wrote:

@thockin https://github.com/thockin we are still interested in finding a
way to solve the underlying use case, where there is an off-cluster service
that we want to manage and treat like an on-cluster service. This meshes
with the service catalog/linking proposal (#17543
#17543), which I still need
to find time to do another update pass to.

I think one of the biggest hurdles is TLS hostname verification if we
proceed with CNAMEs.

—
Reply to this email directly or view it on GitHub
#13748 (comment)
.

On one hand this is orthogonal to .type (ClusterIP, NodePort, LoadBalancer) but at the same time not really. Does it make any sense to have a cluster IP or external access to one of these "imported" services?

I think the import has to be entirely within the Service object, and not in the Endpoints object, otherwise the validation of Endpoints depends on values from Service (you can't have more than one CNAME value for name).

Ideas:

1. Import as a type.

kind: Service
spec:
  type: Imported
  import: google.com

2. Move DNS up and out. Make a new Name object that is what DNS really keys from. Creating a Service would automatically create a Name object. This is imperfect because we already have names across different sub-spaces (svc, pod) but the DNS schema we have has the sub-space as higher-scope than the namespace. If DNS was $svc.svc.$ns.cluster.local then the Name object could be $svc.svc. If we just make Name "extra" names then we have the risk of collisions as Clayton pointed out. This approach is more complex than I am happy about.

I think having a direct CNAME option would be more desirable for cluster users that don;t have access to manage DNS configurations, possibly other use cases. However, I could see how this could get out of control with lots of CNAMES for resources that already exist in another DNS server anyways. I don't think that is what anyone would want. If all we're doing here is providing a way to access services in some DNS system that are outside the scope of skydns but already routable, then why not use the skydns api and add a DNS forwarder? Perhaps a yaml object to add a forwarder would be easier for folks in that case? curl -XPUT http://127.0.0.1:4001/v2/keys/skydns/config \ -d value='{"nameservers": ["xxx.xxx.xxx.xxx:53","xxx.xxx.xxx.xxx:53"]}'

TLS definitely makes this harder.
On Jan 18, 2016 4:46 PM, "Clayton Coleman" notifications@github.com wrote:

Well, the use case is specifically to create local indirection to make
the local application portable. A forwarder configuration is global,
not local, or in the control of the cluster admin, not the local code
that is running inside a namespace.

I tend to agree with Tim that additional names has little value above
and beyond the single name - and also that a service of this type
tends not to be another type at the same time. I an concerned that
imported service varies slightly from referential service if we need
to do something interesting to make TLS useful (my personal bias is
that solving tls referential here is critical).

—
Reply to this email directly or view it on GitHub
#13748 (comment)
.

I agree it is important for this use case, though there's a certain amount
of "left hand better know what right hand is up to". If you're importing
an ELB or something that you control the backend for, make sure your certs
cover appropriate names.

If you're importing, e.g., google.com - I'm not sure what to do with that.

On Mon, Jan 18, 2016 at 8:46 PM, Clayton Coleman notifications@github.com
wrote:

Do you agree it's important? The idea of services abstracting remote
endpoints is (in my head) paired with the ability to trust the remote
endpoint. The age of trusting the network as secure is over. If
services are the foundation of the next generation of apps, not
understanding how transport security becomes integral seems to leave a
large story unsolved.

—
Reply to this email directly or view it on GitHub
#13748 (comment)
.

Can anyone please tell me which release is going to include this feature?

This was in Kube 1.4.

On Mon, Nov 21, 2016 at 1:17 PM, leslie-wang notifications@github.com
wrote:

Can anyone please tell me which release is going to include this feature?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#13748 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p5YTaVw9pbbb-ifS7Qoj2cxD1Jo_ks5rAeBLgaJpZM4F6bVR
.

As this is not obvious from the thread, this is actually tracked here:
kubernetes/enhancements#33