Facilitating the spread of knowledge and innovation in professional software development

Contribute

### Topics

InfoQ Homepage Articles Using DNS for REST Web Service Discovery

# Using DNS for REST Web Service Discovery

The use of the REST architectural style is steadily gaining momentum for both public facing Web services and enterprise integration. However, one aspect of a service oriented architecture does not yet receive sufficient attention: Service Discovery.

In this article, I will describe how existing Web technology can be leveraged to enable Service Discovery for RESTful Web services.

## Looking for a Service Discovery Solution

A client that wishes to interact with a service needs an initial URI to enter the application provided by the service. How can a client get hold of such an entry URI? There are three alternatives:

1. Hard-coding or configuring the URI into the client.
2. Creating a dedicated HTTP-based solution (a service registry Web application).
3. Leveraging some existing service discovery mechanism.

At present, hard coding or configuring the start URI into the client seems to be the prevalent approach. However, the coupling introduced by this practice is a disadvantage because it limits the ability of the service provider to relocate the implementation or to apply certain forms of load balancing.

Providing an HTTP-based service registry by creating a dedicated Web application and associated standard media types is a viable option but should only be the last resort if no existing technology can be leveraged.

Using an existing, already deployed technology is obviously the best solution and we need not look very far to identify a large scale, ubiquitous technology that provides a suitable solution for service discovery: the Domain Name System (DNS). DNS is a distributed database primarily known for the purpose of looking up the IP-address that corresponds to a given host name but it is also used for other retrieval tasks such looking up the mail server responsible for a given domain (DNS MX records).

This article describes how to leverage standard DNS functionality for service discovery.

## Service Discovery Requirements

What are the desired capabilities for a service discovery solution? The common scenario is a client that queries the discovery mechanism for a list of services that implement a certain functionality and then picks one of the returned services for interaction. Usually, the client has the intention to limit the query to some context, for example, a client might want to ask not for all shopping services, but for the shopping services at www.examplebooks.com.

We can break this apart into a list of requirements: A service discovery mechanism should enable the client to

1. Obtain a list of all services of a certain kind.
2. Obtain the entry URI of a specific service instance.
4. Specify the discovery perimeter of interest ("Show me all search engines in the domain example.org").

We'll see shortly that the existing Domain Name System supports all of these requirements. However, we must take a detour first and introduce the notion of service types to RESTful systems.

## Service Types in RESTful Systems

One of the requirements of service discovery is to enable the discovery of a service based on desired capabilities. A client would typically ask for a list of services of a specific kind and then select one of the available services either based on some additional meta data or at random.

Unfortunately, the notion of service types is not yet present in the field of REST Web services and in order to enable type-based service discovery we must first introduce a means to specify service types.

What is the appropriate place to define a service type and its name? Given that the only documentation to be used in RESTful systems is the specification of media types and link relations the natural place to specify a service type is within a media type specification.

How would that work? Let's look at two practical examples concerning blogging services and search services.

## Examples of Defining REST Web Service Types

### Atom Publishing Protocol

The Atom Publishing Protocol specification defines a set of media types and link relations that can be used to implement a publishing interface. The original use case for the specification was weblog publishing, but the Atom Publishing Protocol can be used to create a service interface for any system that manages items and organizes them into item collections.

Defining a service type for Atom Publishing Protocol services would be as easy as specifying a name for that service type inside the Atom Publishing Protocol specification. Clients could then use that name in order to query service discovery mechanisms for services that implement the Atom Publishing Protocol. For the purpose of the article, lets define the service name _atom_http for Atom enabled services. (Do not worry about the underscore or the format now, it is a DNS convention which I'll explain later.)

### OpenSearch

The OpenSearch specification states "The OpenSearch description document format can be used to describe a search engine so that it can be used by search client applications." We need not look far to see the service type implied here: search engines. Turning that into a service type name like the one for Atom Publishing Protocol services yields something like _search_http

Adding this service type name to the OpenSearch specification would be sufficient to use this name for DNS-based service discovery.

## The Domain example.org

I'll use the service types _atom_http and _search_http in a practical example system in order to illustrate the ideas discussed in this article. The diagram below shows the domain example.org. There are two hosts mars.example.org and neptune.example.org on which several Atom Publishing Protocol- and Search services are running.

On the host mars.example.org there are two Atom services that provide access to a knowledge base and a news feed. A Search service provides retrieval access to the news archive. The machine neptune.example.org hosts a couple of blogs and an associated search service provides retrieval access to the entries in the hosted blogs.

According to the stated requirements, a service discovery mechanism should be able to answer the following questions:

• Which services are there that support the Atom Publishing Protocol?
• Which services are there that support search as specified by the OpenSearch specification?
• What is the location of any given service?
• What meta data is available for any given service?

In addition, it should be possible to limit the answer to the domain example.org.

## A (Very) Brief Introduction to DNS

In the early days of the Internet, host names where mapped to IP-addresses in a static file (hosts.txt) located on every host. When the number of hosts on the Internet grew updating this file became a scalability problem. DNS has been created to solve this scalability problem by decentralizing the administration of the database. Decentralized administration is achieved by delegation, which means that the owner of a certain segment of the index path (a domain) can delegate the ownership of portions of that domain to other organizations thereby also delegating administrative responsibility (and burden).

DNS is a path-indexed, distributed database that has the following major capabilities:

1. Distribution - The DNS database is distributed across the many nameserver hosts of the Internet.
2. Delegation - Owners of a certain segment of the index path can delegate ownership of portions of their domain.
3. Typed Resource Records - DNS can associate several kinds of data records with a given domain. Resolvers specify in their queries which kind of data record they are interested in.
4. Caching - In order to improve performance, DNS has built-in caching.
5. Fault tolerance - DNS servers can be replicated to provide continuous service even if individual name servers are not operating.

The administrative owner of a domain operates nameservers that store information and answer DNS queries about the domain. Nameservers typically have complete information about some part(s) of the domain called zone(s).

On startup a nameserver loads the information about one ore more zones from a file (or from other nameservers) and is then able to answer queries about theses zones.

When a DNS client (a resolver) queries DNS for information about a certain domain it contacts its configured "go-to" nameserver which in turn queries other nameservers until the requested resource record has been located and is sent back to the client.

## Leveraging DNS for Service Discovery

DNS can associate various kinds of so called Resource Records with a given domain. In addition to the more familiar ones like A-records (for address lookup) or MX-records (for mail server lookup) DNS defines the resource records types SRV (for service location), TXT (for arbitrary text data) and PTR (for expressing a reference to another domain). A combination of the latter three is used to provide service discovery capabilities with DNS.

## SRV Resource Records

SRV Resource Records are used to express at which host and port within a zone a certain service is accessible. An SRV line in a zone configuration file looks like this:

 _ldap._tcp.example.org.   IN   SRV  0 0 389 venus.example.org.

The configuration line above states that a service of the type _ldap is available at port 389 on the host venus.example.org.

Note: The fields filled with zero can be used for load distribution purposes and the symbolic names _tcp and _udp are a convention specified by the SRV Resource Record specification. They identify the intended protocol and are simply placed between service type and domain.

Using a resolver software like nslookup we can ask DNS for the ldap services available at example.org:

$nslookup -q=srv _ldap._tcp.example.org. Server: xxxx Address: ip _ldap._tcp.example.org service = 0 0 389 venus.example.org.  This call to nslookup issues a DNS query for all available SRV records for _ldap._tcp.example.org. (The command line option -q=srv tells the resolver to ask specifically for SRV records) If there were more LDAP services available, the zone configuration would look like this: _ldap._tcp.example.org. IN SRV 0 0 389 venus.example.org. _ldap._tcp.example.org. IN SRV 0 0 389 mercury.example.org. _ldap._tcp.example.org. IN SRV 0 0 389 earth.example.org.  For this configuration the above query would yield $ nslookup -q=srv _ldap._tcp.example.org.
Server:		xxxx

_ldap._tcp.example.org	service = 0 0 389 venus.example.org.
_ldap._tcp.example.org	service = 0 0 389 mercury.example.org.
_ldap._tcp.example.org	service = 0 0 389 earth.example.org.     

The meaning of this reply is that there are three LDAP servers in the example.com domain along with information about their location (host and port).

## DNS Service Discovery (DNS-SD)

SRV records have a significant limitation regarding the stated service lookup requirements: they cannot be used to configure named instances of a service type and they only support a single service for any given host and port combination. It is, for example, not possible to configure SRV records for two search services that run in the same Web application and therefore share a common host and port. In addition, SRV records do not support the configuration of meta data for a specific service instance.

The DNS Service Discovery specification has been developed to overcome this limitation. DNS-SD combines SRV, PTR, and TXT resource records to meet all the requirements for service lookup. DNS-SD uses the three records types in the following way:

• PTR - used to map service types to named service instances.
• SRV - used to provide location and port for service instances.
• TXT - used to provide additional meta data about service instances.

## Configuring the List of Service Instances - PTR Records

DNS-SD combines service type domain names with PTR records to map service type names to service instance names. This enables the retrieval of the list of all instances of a given service type.

The following zone configuration lines illustrate this idea:

# Note that all names are relative to example.org, for example,
# _atom_http._tcp is really _atom_http._tcp.example.org.
_atom_http._tcp              PTR KnowBase._atom_http
_atom_http._tcp              PTR News._atom_http._tcp
_atom_http._tcp              PTR JimBlog._atom_http._tcp
_atom_http._tcp              PTR MaryBlog._atom_http._tcp
_atom_http._tcp              PTR SallyBlog._atom_http._tcp
_search_http._tcp            PTR NewsSearch._search_http._tcp
_search_http._tcp            PTR BlogSearch._search._tcp  

On the left hand side of each PTR line a service type domain name is given and on the right hand side a corresponding instance of that type.

DNS Service Discovery specifies the following service instance naming convention:

<Instance>.<ServiceType>.<Protocol>.<Domain>

For example, the full instance name for Jim's Blog service would be

JimBlog._atom_http._tcp.example.org

A query for all services of type _atom_http._tcp at example.org would now be expressed as (note the use of ptr instead of srv in the -q option):

$nslookup -q=ptr _atom_http._tcp.example.org Server: xxxx Address: ip _atom_http._tcp.example.org name = MaryBlog._atom_http._tcp.example.org. _atom_http._tcp.example.org name = SallyBlog._atom_http._tcp.example.org. _atom_http._tcp.example.org name = KnowBase._atom_http.example.org. _atom_http._tcp.example.org name = News._atom_http._tcp.example.org. _atom_http._tcp.example.org name = JimBlog._atom_http._tcp.example.org.  DNS-SD interpretes the PTR-query as a query for a list of instances of the service type specified by the lookup domain (here: _atom_http._tcp.example.org). The right hand side of the result therefore provides the retrieved service instance names. Using the service instance names the system setup diagram of the domain example.org now looks like this: ## Configuring Service Instance Data - PTR- and TXT Records The DNS-SD specific use of PTR records enables the DNS client to obtain a list of service instances. Service instances in turn are described using SRV and TXT records. The SRV records provide information about the host and port of a given service instance and the TXT records provide additional meta data pertaining to that instance. DNS-SD specifies a generic key-value format for TXT records (key1=val1,key2=val2,...) and defers the specification of the keys to be used with a given service type to the type's specification. In the case of HTTP-based services such as the examples _atom_http and _search_http at least a path key is necessary to construct the entry URI of a service instance. Here is the example for Jim's Blog service: JimBlog._atom_http._tcp SRV 0 0 80 mars.example.org. TXT path=/blogs/jim  This configuration expresses that the service instance JimBlog._atom_http._tcp.example.org can be accessed on mars.example.org at port 80. The TXT record specifies a path parameter which the client must use for constructing the service's entry http-URI. The path parameter and the fact that the URI must be an http URI must be part of the specification of the service type. Hence Atom services using https would need a different service type, for example _atom_https. Instance information about Jim'S blog service can be retrieved like this: $ nslookup -q=any JimBlog._atom_http._tcp.example.org.
Server: xxxx

JimBlog._atom_http._tcp.example.org service = 0 0 80 neptune.example.org.
JimBlog._atom_http._tcp.example.org text = "path=/blogs/jim"

From which the client can construct the entry URI of the service according to the rules defined by the service type specification. For the given example the resulting service URI would be:

http://neptune.example.org:80/blogs/jim

The following diagram shows the service hosts mars.example.org and neptune.example.org and the nameserver host nameserver.example.org. The large text artifact shows the complete zone configuration file for example.org.

At the top there is a SOA record which indicates that this nameserver is authoritative for the given zone (SOA = "start of authority") and below that we see a couple of lines that define the nameserver (NS record) for the zone and the IP-addresses (A-record) for the hosts used in the example.

The rest of the file contains the configuration for the complete example system as discussed in detail for Jim's blog service. The configuration entries for Jim's blog service are highlighted to show how they refer to one another and eventually to the service instance on neptune.example.org.

## Complete Interaction Example

I have discussed, how DNS-SD uses PTR-, SRV-, and TXT resource records to enable DNS based service discovery. To summarize, the following examples show how a complete lookup process would look like.

1. Retrieve list of Atom Publishing Protocol services at example.org.

$nslookup -q=ptr _atom_http._tcp.example.org Server: xxxx Address: ip _atom_http._tcp.example.org name = MaryBlog._atom_http._tcp.example.org. _atom_http._tcp.example.org name = SallyBlog._atom_http._tcp.example.org. _atom_http._tcp.example.org name = KnowBase._atom_http.example.org. _atom_http._tcp.example.org name = News._atom_http._tcp.example.org. _atom_http._tcp.example.org name = JimBlog._atom_http._tcp.example.org.  2. Retrieve location and meta data of desired instance. $ nslookup -q=any MaryBlog._atom_http._tcp.example.org.
Server:		xxxx

MaryBlog._atom_http._tcp.example.org	service = 0 0 80 neptune.example.org.
MaryBlog._atom_http._tcp.example.org	text = "path=/blogs/mary" 

3. Construct the service entry URI based on the definition of the service type _atom_http.

http://neptune.example.org:80/blogs/mary

4. Access the service via HTTP to obtain initial application state.

\$ curl http://neptune.example.org:80/blogs/mary
200 Ok
Content-Type: application/atomsvc+xml

<service>
...
</service>   

In the example shown above, the client picks any of the service instances returned by the first query. This is only appropriate if the client does not need to differentiate between the instances, for example if the services provide a lookup or transformation functionality. However, in most cases the instances will be significant to the client based on the data they operate on. In such scenarios I expect the service instance configuration to specify the desired service instance name and let the client late bind to the service based on the instance name.

## Overall Advantages of Using DNS

Applying an already existing technology, especially one that has been ubiquitous for over a decade, has numerous advantages, such as:

• A wide range of well tested implementations is available, many open sourced.
• Knowledgeable developers and administrators are widely available.
• Existing ubiquitous use protects investment in the technology.

DNS itself has at least the following specific advantages:

• Reliability through replication
• Built-in caching
• Delegation of administrative responsibility and burden
• Easy to configure
• Well supported (DNS-SD is the basis of Apple's Bonjour Protocol)

## Summary

Service Discovery is an essential aspect of service orientated architecture because it avoids early binding of clients to particular service instances. Removing such coupling provides greater flexibility for reconfiguration of the overall system.

Service Discovery can be easily introduced to systems of RESTful Web services by leveraging standard DNS mechanisms as specified by DNS-SD. DNS based service discovery is readily available to anyone in any system environment given the ubiquitous availability of DNS nameserver and resolver implementations.

By applying DNS to enable Service Discovery you gain the performance and reliability of an Internet technology that has been deployed successfully at global scale for over a decade.

Style

## Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

• ##### DNS already provides the necessary binding from URIs to IPs

by Subbu Allamaraju,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by David Dossot,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Subbu Allamaraju,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Dan Creswell,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Jan Algermissen,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Edward Garson,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Jan Algermissen,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Dan Creswell,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Edward Garson,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Jan Algermissen,

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Edward Garson,

• ##### Discovery Services Are services too

by Reda Bendiar,

by Dan Creswell,

• ##### Re: Half The Answer

by Jan Algermissen,

• ##### Caching Delay

by Stefan Tilkov,

• ##### Re: Caching Delay

by Jan Algermissen,

• ##### Re: Caching Delay

by Jan Algermissen,

• ##### Re: Caching Delay

by Dan Creswell,

• ##### DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

I'm missing the basic premise of this article. Is it to just say that "DNS provides the name-to-location mapping that typical service registries want to provide" and "applications don't need to worry about it" ? But isn't it so fundamental and implicit in the use of URIs, domain names, and DNS on the web - independently of REST and HTTP?

by Dan Creswell,

Your message is awaiting moderation. Thank you for participating in the discussion.

A good article, would like to see some more coverage of registering services in the first place.

Of course this could be effected by hacking around with the DNS server files but that's error-prone and won't scale well (human effort/accuracy wise) as you increase the number of services in terms of e.g. deployment actions. Probably, what you'd really want to do is make use of DNS dynamic update which has it's own challenges like changes having to originate from a specific master-node (what if the master is down?).

Once past this problem steps must be taken to make sure that registered services are appropriately available at the ultimate resolved address. One can use lots of techniques for this (load balancers etc). Last challenge is service failures and relocations that can render details in DNS invalid and require some form of intervention to clean them out. Ideally one would be using something like this: files.dns-sd.org/draft-sekar-dns-ul.txt but it's only a draft at this stage.

Ironically, a lot of this is easier with straight DNS-SD on the local-link network via multicast as opposed to the wide-area/DNS based version.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by David Dossot,

Your message is awaiting moderation. Thank you for participating in the discussion.

Subbu, it seems you're missing the "discovery" part, which goes further than mapping names and URIs. The idea is to use the DNS infrastructure as the meta-data repository required to perform service discovery.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Not sure if that is the most appropriate approach as there are other application/HTTP level ways of providing such metadata. But I would let some DNS experts on comment on the implications of pushing down application-level issues to the DNS level.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Dan Creswell,

Your message is awaiting moderation. Thank you for participating in the discussion.

Not sure if that is the most appropriate approach as there are other application/HTTP level ways of providing such metadata. But I would let some DNS experts on comment on the implications of pushing down application-level issues to the DNS level.

Well, the DNS RFC says in respect of TXT records: "TXT RRs are used to hold descriptive text. The semantics of the text depends on the domain where it is found."

So, it's a policy choice - one which e.g. SPF (www.openspf.org/Introduction) exploits as does ZeroConf/DNS-SD. One complicating factor is that there are some practical limitations on the size of a TXT record:

files.dns-sd.org/draft-cheshire-dnsext-dns-sd.txt (see TXT Record Size)

• ##### Caching Delay

Your message is awaiting moderation. Thank you for participating in the discussion.

My main concern with the idea would be the caching-related delays that one can expect, if domain information changes of other aspects are any indication. Or am I missing something due to my lack of knowledge about DNS details?

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Why do you refer to discovery metadata as being an application-level issue? The discovery metadata is independent of the particular service that is to be discovered. Ideally, the 'path' key would be specified for all service type names starting with _http.

• ##### Re: Half The Answer

Your message is awaiting moderation. Thank you for participating in the discussion.

Yes, it is correct that there are a number of practical issues to be addressed. The point of the article was to put DNS-SD in the context of Web-service discovery to see where that leads as opposed to designing a discovery mechanism as a Web-service itself. The latter would require dedicated media types and corresponding clients while the infrastructure for the former is already globally in place. I looked at DNS mulicast/ZeroConf but choose to keep the article simple and hoped for the comment threads to cover such aspects. Thanks for the link - will need to digest that and maybe the registry maintenance issues make for a good follow-up article topic.

• ##### Re: Caching Delay

Your message is awaiting moderation. Thank you for participating in the discussion.

Stefan,
(I hoped you would raise your editorial remark as a comment :-) I think this can only properly be answered based on practical experience because the partitioning and time-to-live of the configuration might be very different from what is usually done for IP-to-hostname resolution. For example, with DNS-SD you can move the whole _tcp subtree on a dedicated nameserver. Maybe a real-world, enterprise DNS-administration expert reads this thread and can provide some hands-on information?

• ##### Re: Caching Delay

Your message is awaiting moderation. Thank you for participating in the discussion.

In the context of DNS caching issues it is also interesting to look at the approach pursued by CalDAV/CardDAV The I-D combines SRV records with well-known URIs. I have asked the author why they were not using DNS-SD and their concern was caching, he replied.

I think that the I-D does not sufficiently provide for a service instance and type differentiation because it lacks the additional indirection of DNS-SD and I also think that especially caching problems related to the path information can be better solved with HTTP redirection on a per-server basis. However, it is still useful to see how others approach the discovery issue.

• ##### Re: Caching Delay

by Dan Creswell,

Your message is awaiting moderation. Thank you for participating in the discussion.

Having built one of these critters previously, I can say we treated DNS as the provider of several seeds for some service any or all of which which could then be contacted to get the ultimate endpoint for entry into the service.

This works around the caching/ttl problem for the most part as whilst the seeds themselves can't rapidly change, the meta-data they use for redirection decisions can as it's not held in DNS. The trick is to have sufficient seeds that if a few are down, one can still make progress. Note that the number of seeds has little relation to the underlying size of the cluster running a service.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

I'm with Subbu.

Further, this notion of 'service discovery' is a red herring. REST doesn't need service discovery, SOA doesn't either, and it doesn't work in practice. Registries in various guises (e.g. UDDI) never took off because this idea that applications would be able to dynamically discover and invoke services to produce some useful outcome has turned out to be bunk.

URIs offer the requisite level of indirection (i.e. everything required) for operations to effectively manage deployed services and for applications to consume them.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Edward,
in REST-based systems users (human or machine) need URIs to provide to user agents to engage in an application. If such entry URIs are neither hard coded or placed in the configuration a lookup mechanism of some kind is necessary. IOW, if you argue against the need for a service discovery mechanism you are at the same time accepting to either hard code or configure URIs on the client side. While it is possible to argue that this kind of coupling is ok it is still important to be aware of it. I'd say it depends on your requirements whether you can live with it.

Suppose company A is doing some form of automated supply chain management with company B. Does A really want to hard code or configure the entry URI of B's procurement service in its own clients or would A rather do a DNS lookup for the procurement service's entry URI at company-b.com? Would B be happy to have to tell all consumer companies when they change their service URIs? I am not so sure.

Besides entry URI lookup I think that a discovery mechanism can turn out very useful for inspecting what services are available in a given domain.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

by Dan Creswell,

Your message is awaiting moderation. Thank you for participating in the discussion.

Suppose company A is doing some form of automated supply chain management with company B. Does A really want to hard code or configure the entry URI of B's procurement service in its own clients or would A rather do a DNS lookup for the procurement service's entry URI at company-b.com? Would B be happy to have to tell all consumer companies when they change their service URIs? I am not so sure.

The same argument applies to enterprises building collections of internal services. For a small number of services and a few developers, manual configuration is fine. In a system with many services, many development teams, a separate ops group and multiple test environments each with their own copies of the services things are not so easy with a significant number of configuration problems caused by human error/misunderstanding.

Certainly the problems have been bad enough at several companies I've worked for that they've been very happy to see a dynamic discovery solution designed, built and deployed by myself and others.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Jan

in REST-based systems users (human or machine) need URIs to provide to user agents to engage in an application [...] if you argue against the need for a service discovery mechanism you are at the same time accepting to either hard code or configure URIs on the client side.

There is no need for "service discovery", given the inherent properties of URIs and ROA. To "configure" the client side (as you put it), just store the entry URI externally of the system, e.g. in a database.

Consider this: from where do you get the nslookup target (the line, "nslookup -q=ptr _atom_http._tcp.example.org" above)? Another DNS lookup? (Kidding). And how was the "desired instance" (Mary's blog) identified?

You see, *something* has to be explicitly named, because we need to start somewhere. And we need to *name* that something, because we can't look for things without some *handle* to it. It cannot be "turtles all the way down", to make an obtuse reference!

The whole notion of "service discovery" -- DNS or not -- just adds an unnecessary level of indirection to a mechanism that already has a very sound one in place, in my opinion. Please correct me if I'm wrong; I'm learning too: but I'm not yet convinced.
While it is possible to argue that this kind of coupling is ok [...]

It is, because URIs are not an egregious form of coupling (far from it!).

I feel this entire discussion may be summarized as follows: *URIs express location of intent, not location of implementation*. I think that is generally a source of a lot of confusion in ROA. A URI is just a moniker, a pointer to something. It isn't the service: you aren't coupling anything.
Suppose company A is doing some form of automated supply chain management with company B. Does A really want to hard code or configure the entry URI of B's procurement service in its own clients or would A rather do a DNS lookup for the procurement service's entry URI at company-b.com? Would B be happy to have to tell all consumer companies when they change their service URIs? I am not so sure.

Cool URIs don't change; so yes, company A should be more than happy to "commit" to an entry URI from company B. And because company B "owns" the URIs that they publish, they can do anything they like with it after publication (redirect, load balance, ...).

Put another way, I don't see any situation where company B would be absolutely required to change a URI after the fact, because URIs are so tremendously flexibly interpreted.
Besides entry URI lookup I think that a discovery mechanism can turn out very useful for inspecting what services are available in a given domain.

That sounds to me like another addressable resource, not a reason to introduce "service discovery". And by "inspecting", I assume you mean "for reading by humans". My approach would be to use a web page in the first instance, before moving onto anything more ceremonious (like an Atom service document).

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Edward,
yes, cool URIs don't change, but they just might need to (company merge, change of domain name, ..) and when they do, DNS-based help would be nice. But I am mostly with you regarding the service instance name to URI lookup: it is likely that once I have the URI of the instance I'll keep it assuming that it won't change (and it shouldn't).

However, what about finding the service instance URI in the first place? Suppose I am just looking for *any* search service? How do I obtain the list of all search services in a given domain?

It would be interesting to compare a well-known URI (RFC5785) based 'service index' (e.g. using HTML and Atom) with a DNS-SD based solution.

• ##### Re: DNS already provides the necessary binding from URIs to IPs

Your message is awaiting moderation. Thank you for participating in the discussion.

Hi Jan,

yes, cool URIs don't change, but they just might need to (company merge, change of domain name, ..)

I think this is fundamental: URIs needn't change. EVER.

In the case of a company merge, the 'old' URI entrypoints would be kept around. Similarly, a change in domain name should not obviate the old entry URI; rather, the service might gain an additional one (because of course services/resources can have multiple URIs 'pointing' to them).

A realistic scenario where a URI might change is if the company went out of business, in which case you'd be up shit's creek anyway ^_^.
and when [URIs change], DNS-based help would be nice.

As above, I don't see any forseeable situation where the service URI would be MANDATED to change. And even if it did, wouldn't it be easier to just change the entry URI in your config file or database, rather than mess around with DNS shenanigans? It's a lot of ceremony to address a problem that is very unlikely to happen.

This all reminds me somewhat of the question, "what happens if our database changes?". The short answer is "it won't", and cooking up a contingency plan is more effort than it's worth.
it is likely that once I have the URI of the instance I'll keep it assuming that it won't change (and it shouldn't).

I find it very interesting that you state "once I have the URI of the instance I'll keep it". Would that not be a strong indicator that you should just begin with the URI?
However, what about finding the service instance URI in the first place? Suppose I am just looking for *any* search service?

That's just it: I don't subscribe to this vision that we can dynamically weave services together in meaningful (i.e. profitable) ways. If you were looking for *any* search service, then by definition *any* one would do. But you aren't and never will be; the one you will pick will reflect very specific application goals. Maybe you need one with OAuth and to be billed for particular usage patterns; perhaps you need the ability to search within results; you'll require a particular SLA to be in place; and so on.

In short, there are simply too many edge cases, wrinkles and standards to choose from(!) in the context of *opaquely* consuming a service. While it is certainly a clever initiative and laudable goal (and the vision of the original SOA), dynamic service consumption is misguided, impractical and infeasible. And to the best of my knowledge, no-one is really using dynamic service discovery and invocation to the extent that was originally envisaged.
How do I obtain the list of all search services in a given domain?

Yes, exactly: there is no globally-accepted standard to do that. Atom service documents can do that in spades, but again, you can't assume the ubiquity of service documents, yet.
It would be interesting to compare a well-known URI (RFC5785) based 'service index' (e.g. using HTML and Atom) with a DNS-SD based solution."

I'm not familiar with that RFC, but it sounds interesting; thanks for that, I'll check it out.

• ##### Discovery Services Are services too

by Reda Bendiar,

Your message is awaiting moderation. Thank you for participating in the discussion.

Completely agree with Edward. In the "human" web. You look for services in directories like demoz yahoo or other directories or search engines. These are web sites (apps) on plain http, just like the services you are looking for.
I remember my first day in the Web, in front of Internet Explorer 2 . I was like "I want to find some cool guitar tabs, Duh!". And then a guy told me that there were "directory" *Web Sites* you can use to find other types of *Web Sites*. so he typed yahoo.com and i could find what i was looking for.
So yes there has to be a discovery service. And no, it doesn't have to be DNS or some other protocol. A directory service is well...a service. So why add a level of indirection while you can implement it in a restful way. A director service is a rest service whose resources are mainly URLs to other services.
And on the other hand as Edward says "dynamic service consumption is misguided, impractical and infeasible". The only use i could think of it is some client application that would be acombination of like feed readers, google reader for instance : You search for feeds and subscribe to them (discover service and use the service).

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p