Converging API Governance and SOA Governance
Achieving Service Oriented Architecture initiative success requires creating loosely coupled consumer-provider connections, enforcing a separation of concerns between consumer and provider, exposing a set of re-usable, shared services, and gaining service consumer adoption. Many development teams publish services, yet struggle to create a service architecture that is widely shared, re-used, and adopted across internal development teams. Instead of creating consistent service architecture and demonstrating service re-use, teams inadvertently produce Just a Bunch of Web Services (JBOWS) or Just a Bunch of REST Services (JBORS).
A single application often consumes a service, and a spaghetti web of One-to-One connections exists between service provider endpoints and consumers. Many teams find a SOA or REST focus may not improve IT agility, but result in simply swapping out IT toolsets, message formats, and protocols.
SOA governance, API governance, and application governance can stand in the gap and improve architectural coherence.
Services, APIs, and Architecture
When crafting a governance roadmap and best practices for Service Oriented Architecture and RESTful design styles, clearly define how services and APIs both contribute to the overall architecture picture.
On the surface, RESTful APIs are simply a specialized version of web services, and provide similar technical benefits. Both REST API design and SOA service design intend to expose discrete functionality via a well-defined interface. The API endpoint and the service endpoint both serve as a core design unit, and teams deliver technical solutions by accessing, aggregating, composing, and orchestrating endpoint interactions. Successful and scalable API and service-based solutions both require Internet messaging infrastructure, service level management, and security.
SOA and APIs share common integration concerns of endpoint location, message models, security, and quality of service. Managed APIs are ‘SOA done correctly’. A managed API is:
- Actively advertised and subscribe-able
- Available with an associated, published service-level agreement (SLA)
- Secured, authenticated, authorized and protected
- Monitored and monetized with analytics
While services should exhibit managed API characteristics, most service endpoints are deployed on platforms that don’t deliver management characteristics. By applying the API façade pattern, teams can layer network addressable endpoints, monitor usage, enforce usage limits, shape traffic, and authorize consumers.
Converging SOA governance and API governance will enable teams to efficiently acquire both SOA service benefits (re-usable and evolvable implementations) and API benefits (extended reach and decoupled interfaces).
In many organizations, SOA initiatives will deliver point-to-point integration instead of architecture. To achieve the desired architecture goal state, teams must modify existing governance programs to mitigate risk factors and support SOA principles. Delaying service governance often results in non-reusable services, proliferation of multiple business domain definitions, and increasing portfolio maintenance cost.
An effective SOA governance program controls the development and operation of service oriented systems, and teams implement SOA governance using using policies, processes, metrics, and organizational incentives:
- Policies specify the “right” way to do things. They codify laws, regulations, corporate guidelines, and best practices.
- Processes are activities that provide an opportunity to test a project or artifact for compliance with policies, and to make a go/no-go decision. Some processes are automatic and system driven; others require human effort.
- Metrics provide visibility into the governance program, and are required to measure compliance and verify policy enforcement.
- The organization should encourage a culture that supports and rewards good governance practices.
A SOA governance program should provide guidance for the entire service lifecycle, including creation, testing, provisioning, utilization, management, and versioning.
SOA governance infrastructure components provide tools and services that support a governance program. They provide mechanisms to manage and maintain governance policies; they provide mechanisms to impose checkpoints during various phases of the software development lifecycle (SDLC) and verify that services, APIs and/or projects comply with these policies; they also provide mechanisms that support manual and automatic approval and exception processes; and they enable integration with traditional SDLC tools and information technology (IT) management and governance systems.
Some SOA governance infrastructure components address development-time governance, and other components address runtime governance. The service registry component bridges the development and runtime components.
Service Portfolio is [A] in SOA
A key SOA end-goal is a service portfolio exhibiting a clean architecture. Technical and industry vertical business domain models are useful starting points for development of an organization’s unique service portfolio. Service design and interface design are small pieces in a larger set of programs that must be managed in order to establish highly effective IT service delivery, collaboration, and trust.
Initiating complimentary projects such as an application rationalization, business process management or enterprise architecture is an excellent mechanism to understand IT dynamics, decompose the business domain, and determine what software assets (and services!) should support business processes.
API governance is heavily influenced by IT business goals and objectives. Leading API governance platforms provide analytics supporting the assessment of IT business value. The platform should capture service tier subscription information, collects usage statistics, present productivity metrics, and integrate with billing and payment systems.
API governance encompasses API subscriptions and API promotion meta-data. Governance activities managing API promotion meta-data include rationalizing keyword tags used to categorize APIs, and developer documentation content management. The governance process should enforce design-time checkpoints before API publication.
API Economy metrics include adoption and usage. API governance establishes API adoption (by version, by API) and usage (by version, by API) processes, policies, and metrics. By understanding API adoption and usage, API business owners and API architects can intelligently invest future development resources, properly plan API infrastructure scale, and rationalize the API portfolio.
Governance Best Practices
To efficiently and effectively deliver IT solutions, teams must synchronize API governance activities and service governance activities. API management provides a straightforward set of lifecycle stages (e.g., created, published, deprecated, retired, blocked) that may be customized by the development team. Governance registries facilitate service metadata management and governance in design, implementation, test, and run-time operations. Figure 2 below depicts the intersection of the two governance views.
Figure 2: API and Service Activity Lifecycle Views
Common Description Needs
When developers consume, aggregate, orchestrate, or compose APIs or services across organizational boundaries, documentation and contracts describing how to interact with the API or service strongly influence adoption. API proponents eschew heavy weight, formal service interface contracts for human readable documentation. The RESTafarian backlash against schema, description languages, and machine-readable policy is starting to abate.
REST best practices are starting to influence design and description. Whether building RESTful APIs or SOA services, teams should:
- Not publish endpoints exposing rigid data models and requiring complicated interaction patterns
- Document how session context and session state influences interactions
- Document the message model
- Describe usage limits and security concerns (i.e. acceptable authentication credentials, authorization protocols)
- Hide internal details from the external interface
API Documentation and Service Metadata
SOA and API tactics should include defining metadata and policy frameworks that provide the scaffolding required to identify, classify, and describe consumption candidates. Metadata is the design and development time information used to describe a service or API. Policies are used to map metadata to deployed service instances and endpoints. Metadata can span the following concerns:
- Overview (e.g. name, description)
- Lifecycle Attributes (e.g. version, version relationships, lifecycle status)
- Classification (e.g. basic, composite, infrastructure, business)
- Endpoint Deployment Attributes (e.g. protocols, location, WS-* specifications)
- Data Model (e.g. JSON Schema, RAML, XML Schema, WSDL, version, semantics, validation)
- Service Level Requirements and Policies (e.g. availability, capacity, responsiveness, security, transaction rate)
- Mediation (e.g. routing, queuing, caching, transformation)
- Dependency Attributes (e.g. APIs, services, databases, directories, frameworks)
- Physical Instance Dependencies (e.g. application platform, security, management)
- Business Process Model (e.g. UML diagram, business classification)
- Contract information (e.g. consumers, providers, utilization)
- Usage Guidelines (e.g. time of day, availability, # of users. throughput)
- Accounting or remuneration options (e.g. pay per use, subscription, chargeback amount)
Metadata enables formal description of the endpoints and promotes discovery and re-use by other teams. Most organizations don’t start with a full inventory effort, but instead simplify the project's scope. They work towards describing a single candidate service/API that is core to the business, refining their metadata taxonomy, and gaining buy-in from solution teams on possible re-use.
Achieving a productive balance between helpful guidance and stifling control requires understanding impediments and team culture. SOA/API initiatives often stall due to:
- Insufficient education, training, and mentoring
- Mistrust or lack of collaboration across organizations
- Conflicting incentives
- Poor service/API design
- Lack of metrics
- Lack of governance
Guidance is necessary, but if governance processes are onerous, they will cause resentment and resistance. Whenever possible, processes should be seamless and automatic. For example, compliance checking can be performed automatically during code check-in or during service registration. Many policy management systems provide tools for automatic compliance checking. Look for products that seamlessly integrate with developer tooling to make it as simple as possible for developers to identify and fix compliance issues.
Contract management systems can be invaluable in supporting governance of the consumer provisioning process, as well as service lifecycle management, enhancement requests, and service versioning.
Successful Big SOA, Small SOA, API-access, and API-centric Enterprise success is predicated on pervasive adoption, sharing, and re-use across project silos, organizational domains, and development communities. Successful governance initiatives implement the following governance activities:
- Establish design review board
- Tighten control of local class and information models
- Adopt shared funding model
- Implement compliance monitoring
- Create common service level agreement definitions
- Create an inventory of application assets and integration connections
- Implement governance review processes to manage change in the inventory
- Extend software development life cycle governance to embrace service-orientation
About the Author
Chris Haddad leads platform evangelism at WSO2. He lives in Space Coast Florida, where he watches rocket launches, rides the ocean surf, and writes about architecture best practices Chris converges pragmatic hands-on practitioner experiences with IT business goals to communicate strategies and tactics when adopting Cloud, DevOps, and API-centric architecture.
Tom Gilb & Kai Gilb Jan 26, 2015