Application Platform Strategies Blog

Role reversal... Do you think you are an IT person?

Posted by Mike Rollings

Today Richard Watson is posting in the EAP blog so I decided to return the favor and post to the APS blog.  Richard typically speaks to the application platform audience and I speak to our executive advisory audience.  The segregation is based on audience interest and the continuing story that evolves in each blog post.  But, does the segregation have to mean anything? Is Richard an APS analyst or an EAP analyst?  Are you an IT person or a business person?

These questions relate to a discussion we have been having within Burton Group about the personification of IT.  They also relate to the divide between business and technology professionals. When "IT" means the department as opposed to the contribution that information technology provides to the business, then you have personified IT. 

A tale of two perspectives... First, the perspective of businesspeople.  In many organizations saying "IT" inspires the meaning those people in the IT department.  The association of "IT" to the "IT department" evolved from the days of the mainframe.  All the IT resources were consolidated in a mainframe computer—and many times in a single computer room. Centralized planning fostered the personification that "IT" was an autocratic ruler—if you want to play with my toys then you play with them under my terms.  Like some societies, autocratic rule may have worked when the mainframe was the only choice, but it did not work when choice was infused into the environment.

Distributed and decentralized implementation became the norm with the advent of personal computers and local area networks.  Decentralization of computing increased choice—for many businesspeople it was easier to create solutions without the aid of IT. For some the personification of "IT" devolved from the autocratic ruler to being a business roadblock based on fighting the loss of control.  Many IT departments evolved to counteract this perception.  Leadership changes occurred -- new CIO, new regime, new ideas -- but the perception does not change in an instant.  For some IT organizations the new tools opened up the possibility to enable new business capabilities.  For them, the personification became business enabler.

Complexity has also taken its toll on the personification of IT. Many IT organizations have defined their value based on the ability to deal with complexity.  Many times we pride ourselves on how much complexity we deal with.  We may even say "you don't need to know, it is way too complex for you to understand."  The message sent is "if it is complex IT will do it", however the perception can become "if it does not appear to be complex then I don't need to involve them".

Next is the perspective of ITpeople.  The business of IT -- It is likely that you have heard this phrase.  The phrase evolved from a time when many IT departments were failing to manage costs and generate value. They realized that like any good business it is important to focus on efficiency, effectiveness, and the value proposition for your customers.  However, some IT departments took this too literally.  The business of IT became our business is IT.

Sure, some organizations provide IT services to other private or public sector organizations as their core business.  These types of organizations have a business model based on revenue from IT services.  This does not describe the IT departments I refer too.  Even if you are a financial services firm where information technology is embedded into most of what you do, your business is not the sale of information technology services, it is financial services.

Back to the questions -- Are you an IT person or a business person?  Is IT the department or the enabler of business capabilities?  I suggest that whether you are a business or technical professional, we are all businesspeople.

So, is this blog post about role reversal or just a necessary adjustment to perspective? You decide.

Stories from the BPM trenches

Blogger: Richard Watson

For the last 2 months, we have been running a contextual research project, our take on in-depth field research on BPM, as I mentioned here.  It is a fascinating learning experience. Turning the tables on us analysts as we hear stories about how your BPM efforts are progressing is very refreshing.  I couldn’t resist picking out some of the great quotes I've heard to share them here.

Here is a BPM program manager on the activities needed to constantly evangelise in order to get executive buy-in for the program:

"A combination of endlessly and relentlessly communicating about it and demonstrating benefits wherever we could."

The methodology for successful outcomes with BPM is changing, from Taylorist micro-automation, to more collaborative, non-linear, and case management usage patterns.  Using BPM infrastructure at the right level to allow smart people do their work, not get in their way, but get visibility into the results, has been a theme of some of the stories, for example:

"A lot of people treat [BPM] like a production line for a car. If you don't build the car on that production line in the order it needs to go then it's not going to end up the right color for the car the customer ordered.

But the reality is, for example, you see this a lot in insurance, and especially in underwriting – effectively you are doing a sales process, and you are using very knowledgeable people to do the underwriting. If you try to follow a strict process and enforce it you end up actually being able to handle 85% of the work, but it's the low value stuff.  What you try to do then is automate the processes around the high value things, where there's high risk, or huge clients. The moment you try to automate those processes that are really more projects than processes, you end up in big trouble."

This also feeds into customer requirements for BPM tools.  BPM Infrastructure that supports these new ways of work will prosper.  

On the platform side things are also getting interesting, from frustration with products lacking cohesion:

"Anything they are calling a suite is just a grab-bag of their last dozen acquisitions."

Some of the more forward-looking companies are looking for a modular, service oriented architecture to really leverage their existing platform investments:

"The [main] difficultly is to integrate an engine into your platform so you can leverage it. This is something most vendors do not understand…Usually the vendors say, you can import users, import roles, import the organization. Why do you need auditing outside our systems? … In protecting their systems they make integration of the systems so difficult that they don't get leverage from them. In the current market, the leading systems I'm afraid, are architecturally ill.

We will see a 4^th wave of BPM platforms over the next 3-4 years which from the beginning are built for high scalability requirements, support all standards on the market, and different process types. Then you can integrate them fully.  Existing systems do not have a modularity that you need for this."

I told you we were going to challenge the BPM orthodoxy!  In the coming months, I’ll be writing more about these emerging themes: collaborative and social BPM, a modular service-oriented BPM platform architecture and what it takes to get a program going.

If you have BPM experience to share, I'd still be delighted to hear from you.

Burton Group Institute - Improving the Development Process

Blogger: Kirk Knoernschild

At this year's Catalyst conference, I led a workshop on Improving the Software Development Process. The workshop was recorded, and is now available through the Burton Group Institute. I've captured some excerpts from the three and a half hour session, which can be viewed in the video below. We covered a broad range of topics in the workshop, just a few of which I highlight below:

• An optimal team structure for software development
• The impact that organizational structure and dynamics have on a software development team.
• Debunking myths and talking about benefits of agile and iterative development. We also talked about the waterfall mistake, and briefly discussed some lean principles such as building quality in, eliminating waste, and delivering fast.
• The SDLC infrastructure, which included a demo illustrating how we can automate various tasks by leveraging this infrastructure.
• Requirements management, the difficulty in stabilizing requirements early, and emergent requirements.
• Continuous integration, tightening the feedback loop, project visibility and transparency, and the value in having a system that always works.

Enjoy the excerpt!

Application Platform Strategies on tour: SOA and Cloud Symposium: 22-23 October, Rotterdam

Blogger: Richard Watson

Join Anne Thomas Manes and myself at SOA Symposium, co-located with Cloud Symposium, 22-23 October, in Rotterdam, The Netherlands.

On Thursday 22nd Oct, Anne is delivering the second Opening Keynote - "The Resurrection of SOA" at 9am.  At 11.15, I'm presenting "Service Modeling: Making sure your services deliver value". At 13.15 after lunch, you will have to choose between Anne’s "Proving the Business Value of SOA Investments" and my "Platform as a Service: Application Platform Metamorphosis".  Then I'll be rushing along to the SOA Patterns committee meeting for "Candidate SOA Design Patterns Review & Promotion" Session #1.  To wrap up the day at 17.30, Anne will feature on an Expert Panel, "SOA is Dead, Long Live Next Generation SOA".

On Friday 23rd Oct, I'll be presenting "Cloud Application Architecture: Rebuilding applications for the cloud" at 14.15, then doing more SOA pattern shepherding in the SOA Patterns committee session.  To wrap up the second day and the conference at 17.30, Anne and I will both feature on Expert Panels, "Today's SOA Governance Platforms: Are They Mature Enough?" and "Will the Open Cloud Become a Reality?"

(The session names are hyperlinks to the session descriptions)

So as you can see we'll be kept too busy to check out the post-war architecture which makes Rotterdam such as fascinating place.  Imagine for a moment rebuilding an entire city in the 3 decades of the 1950s-1970s.  What would you come up with?

Pity we can't start building a modular cloud application architecture on such a green field site. Hybrid on-premises/off-premises applications architecture will forever be a reality for enterprises. Some organizations are unwilling or legally not permitted to host every application and its data in public or external clouds. The typical chaotic and brittle applications architecture found in most organizations means that a significant fraction of existing applications are not ready to move to the cloud.

We have a very limited number of free guest passes for the conferences still available, each valued at 1,090 Euro.  If you’d like to attend the conference, leave a comment on the blog, and I'll get back to you.  Once the passes are gone, we can still offer a 25% "friend of the speaker" discount. 

REST-* (I've got a bad feeling about this)

Blogger: Anne Thomas Manes

Thanks to Mark Nottingham (@mnot) for alerting me to the REST-* initiative. He tweeted about it last night (this morning his time):

   REST-*? Well, this will end badly... http://bit.ly/cF6CG.

My sentiments exactly. I'm not sure how I missed it. According to the REST-* website, the effort was launched on July 14. (Bastille Day? How ironic.)

Some other comments from the twitterverse:

@psd: amazed how so many people are taking REST-* seriously. It's a brilliant piece of satire.
@jimwebber: Choo choo! There goes the cluetrain... and REST-* missed it.
@ironick: .@distobj REST-* says avoid envelope formats like... wait for it ... Atom. WTF? When did Atom become anti REST?
@distobj: I think a better name for REST-* would be HTTP-* - makes you wonder why they're not doing it at the IETF
@bobinator: @atmanes REST follows WS, CORBA, DCE, etc. they will never learn

It appears that Bill Burke of Red Hat/JBoss is responsible for REST-*. So what exactly is he planning to do? Well, according to the home page:

While REST has gained huge momentum in the SOA community, there hasn't been a lot of standardization of traditional middleware services. The REST-* community aims to introduce new REST-based standards for these traditional services where none exist and provide well-defined guidelines where protocols do exist.

The effort currently lists two specification projects:

• REST-* Transactions: REST-* Transactions is a specification that attempts to define a RESTful interface for transactions.   It describes the interaction between coordinator services and transaction participants as well as how transactions can propagate in distributed applications.  It defines both a 2-Phase-Commit model as well as a Forward-Compensation protocol.
• REST-* Messaging: Messaging encompasses publish and subscribe and point-to-point protocols.  This specification defines a RESTful interface for queues (p2p) and topics (pub/sub).

What really concerns me about this effort is Bill's perspective on REST. As he said in his "What REST has to be" blog post:

I really don’t care in the end if any of the architectural principles of Roy’s thesis are broken as long these requirements [simplicity, low footprint, interoperability, and flexibility] are met.  Pragmatism has to be the most important thing here.  We can’t fall into religious and academic debates on the purity of a distributed interface.

I believe in being pragmatic, but if you don't adhere to the REST principles (everything is a resource with a uniform addressing scheme [i.e., a URL], interactions using representations, uniform methods, stateless interactions, using hypermedia as the engine of state), you won't produce RESTful systems, and you won't attain the desirable RESTful characteristics (scalability, serendipity, network effects, etc) that REST is supposed to enable.

Bill also asserts that AtomPub isn't RESTful because it uses an envelope. This tells me that Bill equates REST with POX rather than with a set of design principles. (i.e., he doesn't get REST.) And he rejects RESTMS (which builds on AtomPub and AMQP) out of hand. His current spec for REST-* Messaging is nothing more than a RESTful facade over JMS. (He says that he plans to publish a new spec next week, though.)

I can see the need for a RESTful push protocol to enable pub/sub. I think RESTMS has some interesting potential. I reject the idea that we must have transaction protocols. A 2PC protocol cannot be stateless. That doesn't mean that you can't manage transactional integrity in REST. It just means that you can't us a 2PC protocol to do it. But there are other ways to ensure transactional integrity.

A more useful effort would be one that defines RESTful patterns that support and enable mission critical capabilities like reliable delivery, transactional integrity, and the like. But please, let's not reinvent CORBA on REST.

Here's hoping the whole REST-* thing just dies out.

Design Principles: guide rails for service design

Blogger: Richard Watson

Clients often ask us for help with their governance processes. We help them design surveys for their development teams to access the maturity of their applications and design practices with regard to sound architecture principles.

Given the problems we in the industry have had producing a formal definition, service oriented architecture (SOA) is best described as a set of design principles that support the goals of building flexible and maintainable systems. The fundamental design principles for SOA are clean separation of concerns, loose coupling, and service-orientation. These principles are not secret sauce to be poured on – if you are building systems that live and breathe these principles – you're doing SOA.

Anne Thomas Manes will publish a report later this quarter detailing the motivations behind, techniques for, and consequences of applying these SOA design principles.

How principled are you?

With any governance processes like these, clearly articulating the motivation for the principles is crucial. No professional, developers especially, like to be told to – just do it – it's good for you. Discovering the impact of abusing the design principles is not difficult to find. Think about these conversations you might have heard in the office.

When your manager asks	Can you answer these tough questions?	They're really asking about
Do you think the clients will mind if we …?	Move the server? Upgrade the interface? Re-implement in Ruby?	Loose coupling
Haven't we done this before somewhere?	Why do I have 4 customer profile applications in my call centre?	Service-orientation
How expensive is it to make changes?	What do you mean changing the UI forces us to refactor the data access layer?	Clean separation of concerns

These are just 3 examples of questions addressed by the SOA principles. If you cannot answer questions like these, you have some work to do in your service portfolio to enforce the principles more cleanly. If we continue to build systems that abuse service design principles, desired outcomes will suffer. If the service granularity is wrong or if interface and implementation concerns are badly separated, then the service will be less reusable. If business logic and infrastructure concerns are blended, developers can't make changes to systems quickly and easily. Tightly coupled systems are going to get in the way of manageability. 

It's the outcomes, stupid

If SOA is about contributing to business outcomes, then measuring the value of services must start with the desired business outcomes. To be credible and repeatable, metrics must be underpinned with measurements from your runtime infrastructure and service repository.  In her forthcoming paper, Anne will frame the principles and techniques with the benefits from applying each principle and with metrics for demonstrating the impact of applying or neglecting each of them. Stay tuned.

Building 2010 Plan

Burton Group's teams are currently formulating research plans for 2010.  Let us know how Burton Group should focus our efforts to provide you with effective decision making tools, perspectives, and insights. 

What initiative or project will be a critical part of your success next year?  How can Burton Group improve our coverage and our research deliverable formats?

Suggestions and comments can be sent to chaddad@burtongroup.com.   Thanks for your feedback.

Cloud vs SOA

Blogger: Anne Thomas Manes

Joe McKendrick posted an article yesterday entitled, "Cloud: the SOA we always wanted, but never had?" He's reporting on a panel discussion he had with Phil Wainewright, Ed Horst, and David Bressler.

Typically I tend to agree with Joe, but not so much this time. I'm more aligned with Mike Kavis on this one.

Mike expressed his distaste for the article in the following Tweet:

@madgreek65 I read this again http://tinyurl.com/mjnar6  Started a comment, then a blog post, then said forget it.  Too much wrong in that post.

I wouldn't go so far as to say that Joe's article is completely wrong. For example, it asserts in the opening paragraphs:

"The consensus seemed to be that cloud is helping to boost the advantages promised by service orientation to a firmer business footing."

I certainly agree with this point, but the article quickly takes a left turn into hyperbole land when it asserts the following:

• Cloud (as SOA should be) is well understood, and often demanded, by the business
• Cloud (as SOA should be) is platform, language, and technology agnostic
• Cloud (as SOA should) provides greater visibility and transparency to actual IT costs
• Cloud (as SOA should) necessitates binding contracts between service providers and consumers
• Cloud (as SOA should be) is based on trust between service providers and consumers
• Cloud (as SOA should) originates from business requirements

As far as I can tell, cloud computing *is* none of these things. It *should* be. But cloud is too nascent for such assertions. Besides, in order to achieve these characteristics in cloud-based systems, organizations have to 1- design them that way, and 2- develop the contracts and trust described. You won't achieve these characteristics automagically just by deploying a system to EC2, Force.com, or some other cloud provider.

Participate in Burton Group's BPM Contextual Research Project

During the second half of 2009, Burton Group is conducting a qualitative research project to assess business, cultural, and technology trends driving business process management (BPM) efforts within the enterprise. We want to understand how organizations are deriving value from BPM. We want to gain insight into the factors that enable them to succeed, and the barriers causing failure. What is the set of product requirements that enable effective and efficient business process design, implementation, integration, execution, monitoring, and management?

A mix of face-to-face and telephone interviews will be the methods used for data collection. Our goal is to have an open-ended dialog with participants from business units and IT groups. The project is open to Burton Group clients and non-clients alike. Some participant roles we are looking to interview include:

· Line-of-business managers

· IT managers, architects and project leaders involved in BPM efforts

· Process and Quality Subject Matter Experts

· Executive sponsors of initiatives related to BPM

We use a research methodology derived from Contextual Design. We conduct a relatively small number (20-25) of lengthy interviews (1-2 hours). Following an interview, we socialize and interpret our notes internally to derive a set of "affinities" -- individual notes, observations, and key insights. After we've completed all the interviews and interpretations, we consolidate the affinities (typically 1000+ individual notes) and look for trends and commonalities.

This effort is being coordinated by Richard Watson, Analyst within Burton Group's Application Platform Strategies service. Information obtained during the interview process will be held in confidence. Any publication or presentation arising from this field research project will not mention any participating enterprise or individual by name without the express written permission of the organization.

Those enterprises involved in the project will have the option to take part in a peer review process prior to publication of the research (e.g., provide comments and factual corrections). Participating organizations will also receive a copy of the resulting research document and have the opportunity to discuss its findings with Burton Group.

This research may challenge perceived industry messaging, but the results will provide deep qualitative information derived by evaluating experiences from multiple customers and finding common patterns.

If you have a question, need additional information, or are interesting in participating, please please leave a comment on this blog post and I will get back to you.

Is there a Cloud programming model?

I have been participating in a lively cloud-computing@googlegroups discussion focused on Cloud programming models.

When considering Cloud Computing’s impact on application development and architecture, I have continually asked my team the following questions:

• What is unique?
• What is new?
• How does Cloud Computing change application development and application architecture?

Existing Cloud Computing's momentum is predominantly focused on hardware optimization (IaaS) or delivery of entire applications (SaaS).Building momentum for Platform as a Service (PaaS) has proven trickier because many 'old discarded practices' are being rejuvenated, skepticism abounds, and developers are focused elsewhere. Does current day technology, new products, and accepted culture mean that Rapid Application Development (RAD) environments and Fourth Generation Languages (4GL) will be accepted by mainstream developers? Some proponents think so, others feel like Bill Murray in ‘GroundHog Day’ and wonder ‘why will the concept work this time?’

Interest in Cloud is providing an opportunity to re-think the application platform, review vendor messages (e.g. Google, SalesForce.com, LongJump, Bungee), and determine how application development and application architecture should morph to support Cloud benefits and characteristics.

A recent newsgroup thread discussed how Cloud will impact programming models. A focus on 'programming models' is a red-herring.The best 'programming models' innovations are unseen and instead rely on inversion of control and/or container-based interception.Vendors are extending infrastructure containers and frameworks to transparently support improved dynamic virtualization and more flexible topology distribution (hopefully without new APIs and languages).

But what is new today? As Greg Pfister states in his post, technology concepts such as virtual containers (e.g. Java JVM and PHP interpreters) and decoupling from operating systems and hardware (e.g. java bytecode) have been around for quite some time. Additionally, 'inversion of control' (IoC), 'configurations instead of code' (EJB3.0 annotations instead of java interfaces), and declarative languages (e.g. XAML) are also well established.

But has the infrastructure evolved to transparently realize elastic scalability and optimize resource allocation? or, do developers need to program to the Globus toolkit, .NET WCF, or Force.com APEX?

A straw-man use case is "Can Cloud infrastructure inherently support elastic, scalable, and resource-friendly execution of Plain Old Code Object (e.g. Java POJO, C# class, Tcl script)?" GigaSpaces is a good example of a product enabling more seamless Java code scalability and lower resource footprints when compared to traditional application servers (e.g. IBM WebSphere, Oracle AquaLogic). Another example, code written in C++ which interfaces to the Win32 or LINUX internals is often tightly coupled to hardware, does not following 'service-oriented design principles' (i.e. loose coupling, separation of concerns, interoperability), and may contain single points of contention(e.g. synchronization points, resource locking). The application either doesn't support parallelism, or supports parallelism single machine specific manner.

A Cloud-friendly application supports dynamic deployment of modular components. Picking up and moving a C++/Win32 application to another machine requires moving several operating system resources (e.g. memory, file handles, threads, inter-process communication objects). Decoupling resources behind a service interface is a more Cloud-friendly best practice.

<>A few use case suggestions:

If end-users rely on Software as a Service, then their development environment must support service interactions and corporate infrastructure policies (e.g. security, reliability, availability, performance) should extend beyond their internal data center and span external service providers. WS-Policy based specification standards are a step in the right direction. Policies must be externalized from application code to enable more distributed infrastructure decisions. Following JSR250 and embedding security policy within Java code doesn't facilitate communicating the security policy statements beyond the JVM environment. You will find numerous examples of both .NET attributes and Java annotations which are deployment infrastructure descriptors. Embedding deployment details in code is an inefficient programming model and not Cloud-friendly. The application design technique often breaks the service-oriented concept of 'separation and concerns'.

If end-users rely on IaaS to deploy applications, then having more insight into the application footprint will enable more intelligent resource allocation decisions. Slimming down the application footprint to handle a single unit of work will enable more requests to be packed into a machine.

SOA is about Architecture

Blogger: Anne Thomas Manes

Thanks to Jeff Schneider for pointing this article out to me.

It's been more than 7 months since I published the "SOA is Dead" post, but responses are still coming in. Most recently (yesterday), Dan Woods published an article on Forbes.com entitled "Reviving SOA". I'm pleased to see that Dan read beyond the first paragraph, and he understands the core message of my post (i.e., "SOA has been disappointing and that services should be a key focus"), he nonetheless fell into a common trap of misinterpretation, as I described in my follow-up post, "SOA Obituary: Misinterpretations and Perceptive Enrichment."

Dan falls into the first and largest camp, "Big SOA is dead, but little SOA will survive." He also has a touch of Camp 2, "REST will fix everything." He claims that "REST" is now more popular than SOAP. By "REST" I can only assume that he means plain old XML (POX) over HTTP, because I can assure you that the REST architectural style is much less understood than SOA. And a recent survey of Burton Group clients (mostly F500 companies) clearly indicates that REST is not yet a popular topic among large enterprises -- as much as I try to promote it.

Dan recommends an incremental approach to SOA: Just build services as you need them on a project-by-project basis, and at some point in the future you can go back and consolidate the services you've built and somehow derive some architectural consistency. Unfortunately, this strategy invariably leads to Just a Bunch of Web Services (JABOWS) because the future consolidation step almost never happens. The result is too many services, too many moving parts, and too many brittle connections. Systems wind up being more expensive and more fragile than ever before. #FAIL

I've only encountered one company to date that has succeeding using this method -- and I must point out that they maintained strong architectural governance of all services as they were built.

In particular, Dan misinterpreted my comment, "If you want spectacular gains, then you need to make a spectacular commitment to change." He took this as a recommendation, when it was really intended as a warning.

One of the biggest reasons that SOA "died" is due to unrealistic expectations. The hype has suggested that all you had to do was deploy an ESB and use web services and miraculously you would gain reduced costs and increased agility. The truth, though, is that architectural improvement does not come for free. It requires a concerted effort to improve the application architecture. If you aren't willing to make that concerted effort, you won't gain spectacular results. 

The Bottom Line:

The highly-touted benefits of SOA derive from architectural improvement, not from any particular technology. But architecture is hard. And it requires a fair amount of organizational maturity to accomplish. Organizational maturity is a function of five organizational dynamics:

• Governance: Establishing policies, guidelines, and decision-making authority
• Management: Enforcing policies and directing day-to-day operations
• Leadership: Inspiring people to perform
• Skills: Ensuring people know how to do what they need to do
• Practices: Establishing structure that helps people do what they need to do

Attempting to do SOA without also attempting to strengthen these five organizational pillars will invariably result in failure. Hence my warning: If you want a spectacularly successful SOA initiative, you must first address these organizational and cultural issues. Strengthening these organizational dynamics invariably requires a spectacular commitment to change.

VMWare fuels SpringSource's stratopheric journey

For the last year, the SpringSource team has been intently focused on bridging the gap between development and operations. The team has also been moving quickly to re-tool their platform to become a Cloud-native. Cloud-native platforms are built to exhibit cloud computing characteristics (i.e. self-service, elastic and scalable, virtualized, and dynamic) compared to the 'earthborn migrant' runtime stacks which are simply traditional application platforms hosted on cloud infrastructure without application container and framework modifications.

Cloud-native platforms enable users (including non-developers) to customize pre-built templates. They offer abstractions for workload granularity that lighten the developer’s burden for many scalability and operational concerns. While Cloud-native platforms will attempt to transparently address scalability and elastic provisioning/de-provisioning, the platform will assume that an application is already horizontally scalable, or can be redesigned to conform to Cloud application architecture restrictions that facilitate a scalable solution. Moving to Platform as a Service forces architects to fundamentally re-examine their application architectures.  Tackling parallelism and single points of bottleneck (usually database access) will become paramount.

With a new programming model, developers can approach solving problems that are resistant to traditional application architecture practices.  Democratizing the process of building massively scalable web applications and delivering high-quality applications in a superfast time to market are two examples that are becoming mainstream concerns. SpringSource, a framework innovator, is well-positioned to address the programming model evolution.

Attenuating the feedback loop between VMWare's vSphere and SpringSource's application servers will enable finer grained application footprints and timely elastic scaling. Imagine the ability to dynamically provision a specialized stack (i.e. container, framework, application meta-data and code) to handle a discrete unit of work (e.g. web request, transaction, resource query), intelligently pool the stack, and tear it down when not needed. SpringSource's investment in OSGI, inversion of control, lightweight containers, and application management interfaces establishes a necessary foundation to re-think and shrink run/manage away from monolithic systems (e.g. hardware virtual machines, application servers, and applications) and towards discrete business services.

Service models: Building services that add value, not follies

Blogger: Richard Watson

Should I use WS-* or REST? Should a service provide access over HTTP, MOM, or XMPP? These are the wrong questions for architects to ask when first conceiving a service. By concentrating on how to build, we lose focus on what to build.

Debates about whether to use REST or WS-* interface styles are seductive. But, these are the wrong questions to ask first. Interface style and middleware decisions should be based on consumer requirements. The primary service design principle emphasized in a service model is maintaining a clean separation of capability ('what') from interface concerns ('how'). If we are arguing at all we should be arguing about what services to build, and why.

What is a service? This question cannot be answered in the abstract. Service modeling is an entirely contextual activity. 

Despite their rich history, service design practices remain nascent. The identification and definition of sharable and reusable services is still more art than science. In this paper (Burton Group subscribers only) published today, I examine the neglected art of service modeling. When people hear "model" they immediately think of formalism, notation, and tools. My definition of service modeling is not a formal model or modeling notation. It's thinking about the answers to these questions in a repeatable way.

Can you answer these questions? Do you have places in your service development process where these questions are answered in a consistent way? Service modeling is a set of software architecture practices for providing us with a way to get at answers to these questions and document them in a way that's valuable for everyone who has a stake.

Near where I live, in Dublin, is an extensive park called St. Anne's Park. It's part of what was the Guinness estate (yes, that Guinness!). This is a Greek Temple, built in the park as tea house by the Guinness family. It’s has very ornate mosaics on the floor, and is almost perfect classical design.

But, it's a Folly. Follies are buildings, or parts of buildings that have no purpose other than as an ornament; they are purpose-built as ornaments; they are often eccentric in design or construction; there is often an element of fakery in their construction.

The Guinness family built this because they were wealthy, just because they could. We can't afford to do that in our organizations any more. What we build must be relevant, driven by the right business context, and play its part in a portfolio. As you can see from its state of repair, there's no great motivation to maintain the tea house. It’s not valuable any more, if it ever was...

Put simply, if you create services without a service model, then you are creating bespoke application integration interfaces and not doing service oriented architecture. If context is not driving you to create the right services, then they are most likely not adding value to your applications architecture, they are making it worse.

Let me know what service modeling means to you. If you’re going to the Burton Group Catalyst conference next week in San Diego, I'll be giving a session on service modeling on Wednesday afternoon in the "Whither SOA: What’s next?" topic.

Navigating the Myriad SOA Standards and Specifications

Blogger: Anne Thomas Manes

This just in:

"Members of OASIS, OMG, and The Open Group announce the publication of a new white paper, 'Navigating the SOA Open Standards Landscape Around Architecture.'  The paper was produced to help the SOA community at large navigate the myriad of overlapping technical products produced by these organizations with specific emphasis on the “A” in SOA; i.e., Architecture.

This joint white paper explains and positions standards for SOA reference models, ontologies, reference architectures, maturity models, modeling languages, and  governance. It outlines where the works are similar, highlights the strengths of each body of work, and touches on how the work can be used together in complementary ways. It is also meant as a guide to users of these specifications for selecting the technical products most appropriate for their needs, consistent with where they are today and where they plan to head on their SOA journeys."

Join me at "SOA: The Night of the Living Dead"

Blogger: Anne Thomas Manes

Calling all Boston-based SOA zombies!

My friends, Ron and Jason, over at ZapThink, will be hosting a soiree to discuss the death of SOA.

The venue: MJ O'Connor's Irish Pub on Columbus Ave in Boston, on July 23. Pundits joining me at the event include:

• Ron Schmeltzer
• Jason Bloomberg
• Dana Gardner
• Brenda Michelson
• Sandy Rogers

Total cost: just $29, and that includes food and drink.

For an even better rate, use discount code PRTDISC

Click here to register.

This Imperial Middleware will be operational as planned

Blogger: Richard Watson

Dum Dum Dum Dum, Dum Dum Dum, Dum Dum Dum.

KURIAN*

Lord Phillips, this is an unexpected pleasure. We're honored by your presence.

PHILLIPS**

You may dispense with the pleasantries, Thomas. I'm here to put you back on schedule.

Thomas turns ashen and begins to shake.

KURIAN

I assure you, Lord Phillips, my men are working as fast as they can.

PHILLIPS

Perhaps I can find new ways to motivate them.

KURIAN

I tell you, this middleware will be operational as planned.

PHILLIPS

The Emperor does not share your optimistic appraisal of the situation.

KURIAN

But he asks the impossible. I need more men.

PHILLIPS

Then perhaps you can tell him when he arrives.

KURIAN (aghast)

The Emperor's coming here?

PHILLIPS

That is correct, Thomas. And he is most displeased with your apparent lack of progress.

KURIAN

We shall double our efforts.

PHILLIPS

I hope so, Tom, for your sake. The Emperor is not as forgiving as I am.

<fades>

Commander Kurian will be relieved that it seems the Oracle Fusion deathstar will be operational as planned, as Anne reports here. So, let's award the middleware team 10 out of 10 for execution. Ramming acquisitions into a (excuse the pun) coherent set of products is tough engineering and marketing work.

Product strategy is a different matter. Burton Group's position on heterogeneity (not homogeneity) as a driver for an application platform strategy is well known. We see opportunities for Rebel incursions.

Also, technically, I don't (yet) see any influence of the BEA microkernel architecture surfacing in other parts of the suite. This is a pity, because promoting a modular architecture like OSGi really helps developers digest parts of the stack they need and remove the bits they don’t. The WS02 carbon project and Paremus Service Fabric are pioneers for this type of architecture.

Miko Matsumura has also picked up on the cinematic sweep of Oracle's advances through the middle(earth)ware market.

* Thomas Kurian - Senior Vice President, Oracle Fusion Middleware.

** Charles Phillips – President, Oracle.

Thoughts on Oracle 11g Announcements

Blogger: Anne Thomas Manes

By invitation, I went to a special NDA analyst briefing last week on the Oracle 11g announcements. As I tweeted at the time, I found it very refreshing to hear a strong and definitive strategy underlying the announcements. It's a four-part strategy:

Complete: Oracle wants to be a one-stop shop. You can buy everything you need: applications, software infrastructure, tools, databases, management, and hardware infrastructure from Oracle.

Integrated: All components of the complete platform are designed (or perhaps refitted) to work with each other.

Best-of-breed: Each component in the integrated, complete platform is a credible, competitive product in its own right.

Hot-pluggable: The environment is standards-compliant, so, if you desire, you can replace a best-of-breed Oracle component with a comparable standards-compliant component from another vendor.

Unfortunately, it's a specious strategy.

Yes, the Oracle environment is complete, integrated, and composed of (for the most part) standards-based, best-of-breed offerings. But if you take advantage of the "hot-pluggability" feature, you break the "integrated" benefits of the environment, which derive from the common development and management systems (JDeveloper and Enterprise Manager). But Oracle has deliberately limited the scope of these products to work only with Oracle-supplied platform components.

As alluring as the one-stop shopping strategy is, organizations must learn to just say "no".  The reality is that no one has an entirely homogeneous environment. Oracle claims that Enterprise Manager supports end-to-end business process monitoring, but the concept breaks down if the process includes a .NET service or a third-party COTS application. A better solution is a management strategy that embraces diversity.

Diversity in IT systems is a fact of life. The trend toward heterogeneity is only going to increase as organizations take advantage of cloud computing or implement ebusiness collaboration systems.

As for the specifics of the announcements: Two things I really liked:

• Integration of TopLink and Coherence -- you can now use the Coherence distributed data caching system within TopLink
• Integration of Collaxa and Fuego runtime engines -- a single engine can now run both BPEL scripts and BPMN models

Don't fixate on service reuse

Blogger: Richard Watson

Measuring service reuse alone is not the right way to determine the value of a service.  SOA advocates tend to fixate on service reuse, and fill business cases with return on investment calculations based on reuse projections.

"Reusability is always overpromised."

This is a comment from the CIO of a Fortune 500 company interviewed by Burton Group. It reflects the risk in fixating on reuse. The memory of his application architects claiming that reuse is virtually guaranteed by object-oriented design has not quite dimmed. Likewise, applying service oriented principles promotes reuse, but does not guarantee it.  Reuse is a complex issue, as much cultural as technical.  As Jeremy Meyer eloquently says [i]:

Reuse is about people and education, not just architecture ... The truth is that even the most beautiful, elegant and re-usable architecture, framework or system will only be re-used by people who: a) know it is there, b) know how to use it, and c) are convinced that it is better than doing it themselves.

A service may never be reused, but still be used to create value in other ways: by being adaptable and less costly to maintain, reducing redundancy, increasing security and compliance through consistent enforcement of policies, to name just a few other desirable outcomes. Exclusive focus on reuse blinds us to these other outcomes.
I've had some great conversations recently with clients about calculating reuse. It's a tricky calculation.  Most reuse savings formulae look something like these:

Savings = Cost of Implementation - Cost of Integration
or
Savings = services * reuse * change
or
Savings = sum ( f (cost of integration at consumer, cost of change to service provider for new consumer) ) over #reuse, where f is "some function of".

Reuse savings are problematic because stated like this, as the client I spoke to says, "if I've saved all this money, where is it?"  Of course, it's deferred expenses, but that doesn't sound quite so valuable.  These formulae also overstate the value of reuse over time.  Some kind of decay function is needed to take account of a discount in the 'Cost of Implementation' value.  I'm out of college too long and not crazy enough to attempt the math for that decay function, I just know it exists. I'm not convinced it's always a decay function either.  If a service can take advantage of network effects then its reuse benefit over time actually rises.  And unlike radioactive decay, the value of the service can be refreshed occasionally, say when a changes to reporting regulations dictate a different set of rules and the changes required can be made in an isolated place instead of across the board.  But that gets us back to the value of service "use" not "reuse".

  

[i] Jeremy Meyer. In "97 things every software architect should know", ed. Richard Monson-Haefel.  Sebastopol: CA: O'Reilly. 2009.

News Flash: Insourcing Fosters Relationship

Blogger: Kirk Knoernschild

A recent article on TechTarget claims that IT insourcing is on the rise. In other words, organizations are beginning to bring their recently outsourced IT functions back in-house. The article cites a few motivating forces for this change. Organizations have more control, can respond more quickly to business change, and are leery of outsourcing going forward due to unpleasant past experiences. The articles cites on of the reasons for outsourcing - it was cheaper - may not actually be true. While outsourcing may be cheaper short-term, the long-term cost may be more. Not an epiphany, for sure.

But what really caught my eye was the following quote from the article:

There's a sense of intimacy with the business users that's hard to achieve with outsourcing.

That's a rather profound statement. It took outsourcing for us to realize that collaboration with business users is a critical success factor on an IT initiative? Wow.

IT Labor

Since we're on the topic of outsourcing.... Another popular reason why organizations outsource is due to the IT labor shortage. Many claim there is a lack of qualified IT candidates. This reason is also the foundation of the H-1B program. Not so fast. I've never actually believed this was entirely true. My good friend Randy once said, "There's also a shortage of $2.00 per six pack micro-brews." If you're interested in IT labor, this Lou Dobb's video might interest you.

Conspiracy Theory

I take my car to a certified GM dealership, even though it's a bit more expensive than going to the hometown mechanic. Why? Because I know they won't use secondhand or refurbished parts. I also know they have the specialized equipment and expertise to work on my model vehicle. But if the GM dealership charged a 50% premium for the same level of service I could get at my local auto shop, I'd have no motivation to take my vehicle to the dealership. Now, apply this analogy to corporate IT.

If corporate IT charges a 50% premium over an outsourcing provider, but doesn't provide any strategic or competitive advantage to the organization, what's the motivation to use corporate IT? Instead, it's sound fiscal strategy to outsource. In other words, why should I pay $100 an hour for crap when I can get the same crap for $75?

For now, let's ignore the crap (that's a topic of discussion for another day), and instead focus on cost. $75 an hour may seem cheaper than $100 an hour, but only when you fail to factor in the number of hours it takes to complete a project. And the challenges inherent to distributed development, contract negotation with outsourcing providers, and more will impact delivery, which will increase time, which will drive up cost. So now, we're starting to realize that outsourcing isn't necessarily cheaper. Don't be surprised to see aggressive legislation on the table soon asking for an increase to the H-1B cap. Wouldn't it be ideal to have the best of both worlds? Cheap labor that's co-located! Really? Is it possible there's a causal relation between IT failure, motivation behind outsourcing, and the H-1B program?

Disclaimer: I'm not claiming that everything we deliver is crap. But there is no shortage of evidence illustrating some of the challenges we've had in delivering IT projects.

Disclaimer: I don't claim that we should abolish the H-1B program. However, there is evidence that H-1B visa holders are paid between $12,500 and $20,000 less than citizens. That's not the purpose of the program.

Cloud Computing is "depressingly" similar to SOA

Blogger: Anne Thomas Manes

David Linthicum wins the "grumpy architect" award for today.

He posted a rant on Sys-Con Media entitled, "Getting the Links Straight Between Cloud Computing & SOA". It's well-worth the read. In it he references an ancient (Feb 17, 2009) article by Stacey Higginbotham entitled, "HP Confines the Cloud for Enterprises," in which she wrote up HP's strategy announcement on Cloud Computing.

In particular, Dave was ranting about this comment:

But aside from some vague nods to the benefits of accessing information in the clouds (such as with web-based email) most of HP’s detailed talk of clouds in the first webinar was depressingly similar to the idea of service oriented architecture. HP offered clouds as merely a means to deliver IT as a service inside the enterprise.

Well, Duh!

Alright. I admit it. It's all my fault. I proclaimed that SOA is dead, and people who can't be bothered to actually read the blog post (or any of my follow-on blog posts), took it to mean that we shouldn't bother doing service orientation anymore.

I tried to be really explicit in my last blog post, "SOA: It's Dead, Jim," but perhaps people still didn't get it:

"SOA" as a term has lost its luster, but "SOA" as a practice is essential for all organizations going forward.

As I said in the original obituary:

Service-orientation is a prerequisite for rapid integration of data and business processes; it enables situational development models, such as mashups; and it’s the foundational architecture for SaaS and cloud computing.

The network is a bit overrun with articles talking about the intersection of SOA and Cloud Computing, so it really should be obvious by now: Cloud computing requires SOA.

Note, though, that the exigencies that require we do SOA will do nothing to revive it in the eyes of business people that control its funding. SOA is still dead: business people believe that SOA is expensive, and it doesn't deliver, and they don't want to allocate any more money to it this year. (In case you missed it, this is what I mean when I say, "SOA is dead".)

Our challenge as architects is to figure out how to do SOA without a big pile of money to fund our efforts. Fortunately, SOA doesn't need to cost a lot. It just requires good architectural practices. SOA is about design, not about technology.

Lessons from frugal innovators

Blogger: Richard Watson

I hate putting newspapers and magazines into the green recycling bin before I've extracted every bit of written goodness from them. Unfortunately, I often have to, because my appetite for reading matter is bigger than my slice of reading time. Sometimes, though, I get to look back at a few weeks of the Economist, to make sure I've not missed something. This week my frugality was rewarded, as I came across a gem I'd missed first time around.

 This article [1] is about lessons we can learn from India's healthcare system. The first lesson we learn is from Vivek Jawali, a heart surgeon, and his team at Bangalore's Wockhardt hospital. They have pioneered "beating heart" or open heart surgery with the patient remaining awake for procedures including complex bypasses.

Because such "beating heart" surgery causes little pain and does not require general anaesthesia or blood thinners, patients are back on their feet much faster than usual. This approach, pioneered by Wockhardt, an Indian hospital chain, has proved so safe and successful that medical tourists come to Bangalore from all over the world.

Unlike the hidebound health systems of the rich world, he says, "in our country's patient-centric health system you must innovate." This does not mean adopting every fancy new piece of equipment.

This resonates clearly with the direction of my research this year. In the reincarnation of service oriented architecture, service modeling and other concrete architecture practices are the focus, not buying shiny new equipment. Service architecture innovation involves identifying services in a rich business context to prove their value. Adhering to the proven design principles of separation of concerns, loose-coupling and service-orientation to make sure they deliver value over the longer term by being flexible and maintainable.

Just as I found value in reviewing back issues of the Economist, my advice to our clients is to look back over their application platform infrastructure to squeeze the most from it. I spent much of the winter adding to Burton Group's Reference architecture for service infrastructure. I added decision making tools for service containers, service mediation systems and middleware. The overarching recommendation in each case is to use what you already have, as it probably fulfils 80% of your requirements. This means keeping the operating theatres running more efficiently, rather than building newer gilded halls. Concentrating on patient outcomes, not polishing the marble, or buying a new machine that goes "ping".

Over the years [Dr. Jawali] has rejected surgical robots and "keyhole surgery" kit because the costs did not justify the benefits. Instead, he has looked for tools and techniques that spare resources and improve outcomes.

...

Shivinder Singh, head of Fortis, a rival hospital chain based in New Delhi, says that most of the new, expensive imaging machines are only a little better than older models. Meanwhile, vast markets for poorer patients go unserved. "We got out of this arms race a few years ago," he says. Fortis now promises only that its scanners are "world class", not the newest.

This is the key lesson from the Indian healthcare providers. Patient-centric and outcome-oriented innovation plays to their strengths and allows more patients to be treated at lower cost. Building our IT around patient (business-) centric treatment  rather than technology is the only way enterprise IT can stay relevant, competitive, and innovative.

I like this article because it reminds us in this flat world[2] innovation is a global two-way street. Especially in the IT industry, we tend to view India through cultural cataracts as primarily an outsourcing location, rather than a centre for business and technology innovation we can learn from.

There's another twist in the Economist article. In claiming advances in Indian care have been enabled by adopting healthcare information technology (HIT) the author is on shakier ground. A glance at the blog entries from my colleague Joe Bugajski and the comments from healthcare professionals will confirm that those advances are coincident, not causal.

[1] "Lessons from a frugal innovator". The Economist. 16 Apr 2009.

[2] Thomas L. Friedman. The World Is Flat: A Brief History of the Twenty-first Century. New York, New York: Farrar, Straus and Giroux, 2005

SOA: It's Dead, Jim!

Blogger: Anne Thomas Manes

My pronouncement of SOA's death back in January sparked quite a lively debate. According to Joe McKendrick, the debate is still raging.

Just in case anyone is still confused by what I said/meant when I said "SOA is Dead; Long Live Services": "SOA" as a term has lost its luster, but "SOA" as a practice is essential for all organizations going forward. Many organizations have invested millions into SOA, and they have little benefit to show for it. Some organizations are worse off than when they started. Given the tight economy, business people aren't particularly interested in pouring more money into what looks like a sinking ship. If you want to get funding this year for your SOA initiative, you should probably avoid using the word "SOA" and instead focus your efforts on building "services" that deliver measurable value to the business.

Now, that's not to say that no one has succeeded with SOA. IBM and Software AG both trotted out a number of great success stories this month at their conferences, IMPACT and SOA Summit respectively. In particular, everyone was raving about the Coca-Cola case study at SOA Summit. Burton Group also has some great success stories on deck for Catalyst 2009. But true SOA success is hard to come by. Our definition of "success" is positive return on investment. If you've invested $5 million over 5 years, your initiative is not successful unless you've generated >$5 million in positive business outcomes. 

Which brings me to the real topic of this post: Proof that SOA is still dead.

According to a recent Gartner survey, 40% of users do not measure how long it takes to achieve a return on their SOA investment. The survey also shows that 50% of those companies that have not yet started a SOA initiative did so because they could not articulate and demonstrate its business value. Without a means to measure value, SOA initiatives are doomed. Quoting from the IT PRO article summarizing the Gartner survey:

Massimo Pezzini, research vice president and fellow at Gartner, said that many companies were approaching SOA projects with excessive expectations and little awareness of the effort, resources and time needed to achieve any benefits.

“Some SOA projects are perceived to have failed when in fact there are simply no well established metrics to evaluate success,” he said. The pressure of such expectation, coupled with the promises of SOA technology vendors, were leading companies to over-spend on technology but under-spend from an organisational and governance viewpoint, Paolo Malinverno, research vice president at Gartner added.

“So they come to the conclusion that SOA is expensive and doesn’t deliver,” Malinverno said.

As David Linthicum said in his commentary on the report, "Shame on you guys!"

There was a bit of discussion in the Twitterverse yesterday among @madgreek65 @richardveryard @neilwd @davidlithicum @jhurwitz @Lowrain @cobiacomm and myself (@atmanes) on this report. I loved Neil Ward-Dutton's characterization of ROI measurement as a "minority sport". David has written quite a bit on measuring SOA ROI. As he said in one of his tweets, "Google SOA, ROI, and Linthicum". Burton Group subscribers should also take a look at "Building the Business Case for Service Oriented Architecture Investment". I also have a document scheduled to be released next month on developing business value metrics: "Using Metrics Effectively: Proving and Improving the Business Value of IT". We'll also be talking about business value metrics at Catalyst. We have a few good case studies to share.

I've seen two other recent indications of proof that SOA is still dead:

1. Gartner just recently published its annual assessment of the application integration and middleware (AIM) market, which experienced single digit growth in 2008. According to a review of the report by Application Development Trends, "Middleware Market Hits the Brakes in 2009", Gartner is projecting a 0.8 percent decline in the AIM market for this year.

2. Gartner reports that IBM holds 30.8% of the AIM market. Another market study by Report Buyer asserts that IBM holds 70% of the SOA infrastructure market. So IBM sales should be a pretty good indicator of the SOA infrastructure market. And according to a tweet from James Governor at IMPACT, "Robert LeBlanc GM, software sales says clients are buying SOA in smaller chunks now." (I interpret "buying SOA" to mean "buying SOA infrastructure software", because we all know that you can't buy SOA.)

Now, of course, plenty of people continue to refute my claim that SOA is dead. First in line is Steve Mills, Senior VP and Group Executive of IBM Software. Joe McKendrick asked him for his take on the debate while he was at IMPACT, and then faithfully wrote up his response. Steve vehemently supported my claim that SOA as a practice is essential going forward, but he said nothing to refute the claim that business people have "come to the conclusion that SOA is expensive and doesn’t deliver.” Robert LeBlanc's report on SOA infrastructure sales demonstrates that IBM is certainly feeling the effects of business people's disillusionment with SOA.

Other market sizing research firms disagree with the Gartner report. For example, the Report Buyer survey cited above predicts that the SOA infrastructure market will grow at an average rate of 17.1% over the next 6 years. Assuming the economy recovers within the next 2 years, this prediction seems reasonable -- but the growth will almost certainly be back-loaded. I seriously doubt that we will see double digit growth this year or next. I think Gartner has a better view of the more immediate future.

My friends over at Forrester have also refuted my claim--sort of. Randy Heffner published a document last week entitled, "SOA is Far From Dead, But it Should Be Buried." The title is a bit misleading. When Randy says SOA should be buried, he means that it needs to be "buried inside a larger vision". Actually, I think I said that in the original SOA Obituary:

Successful SOA (i.e., application re-architecture) requires disruption to the status quo. SOA is not simply a matter of deploying new technology and building service interfaces to existing applications; it requires redesign of the application portfolio. And it requires a massive shift in the way IT operates. The small select group of organizations that has seen spectacular gains from SOA did so by treating it as an agent of transformation. In each of these success stories, SOA was just one aspect of the transformation effort. And here’s the secret to success: SOA needs to be part of something bigger. If it isn’t, then you need to ask yourself why you’ve been doing it.

The latest shiny new technology will not make things better. Incremental integration projects will not lead to significantly reduced costs and increased agility. If you want spectacular gains, then you need to make a spectacular commitment to change.

Randy does refute my claim that organizations are reducing their SOA investments, and he backs it up with data from a survey of 2,227 IT executives. Joe McKendrick summarized the report here. According to the survey, 24% of users say that SOA has "delivered most or all of the benefits expected", and 36% say it has "delivered enough of what they expected to justify expanding their SOA adoption". And more to the point, only 1% say they have “seen little or no benefit” and are cutting back on SOA efforts. I presume that the remaining 39% have realized modest benefits at best, but expect to maintain current investment levels.

A 24% success rate is a little higher than what we have directly observed, but not horribly out of line. Besides, Burton Group doesn't run statistically relevant surveys. The 1% "cutting back on SOA efforts" is much lower than our observations, though. Many of our clients (Global 2000 companies and government agencies) have reduced their SOA investments this year. It also contradicts the drop in sales reported by IBM and predicted by Gartner.

I'd like to see how the middle groups (75%) in this survey correlate with the 40% of users that aren't measuring ROI. We've found that many organizations can't definitively say how well their SOA initiatives are going because they lack hard metrics and baselines.

It's quite possible that these organizations have reduced spending on SOA infrastructure, and what they mean is that they are applying SOA practices in a larger percentage of projects. That's what John Rymer implied at his keynote speech at SOA Summit. As reported by Joe McKendrick:

In his presentation kicking off the summit, John Rymer said that Forrester’s surveys show plenty of strength in SOA adoption plans — for example, 27% of the largest enterprises currently have SOA in place, and 33% are committed to moving in this direction. SOA principles themselves did not die, but rather, “SOA died a marketing death,” meaning that the approach has become so vital and basic to enterprises and as a part of packaged applications that marketers have moved onto the next big thing. “When a technology becomes vital, it dies in a marketing sense,” he explained. “It’s time for SOA to ‘die’ since it’s not distinguishable anymore since everybody’s using it.”

I certainly hope John's assessment is true.

SOA is still the most popular search term on the Burton Group research site. And we're definitely very busy assisting clients with their SOA initiatives. But we're still seeing a lot more stalled efforts and failures than success stories. So I don't think we're out of the woods yet.

Increasing Your Agility

Blogger: Kirk Knoernschild

Some remark that agile development processes have crossed they chasm - agile is mainstream and adoption is widespread. I'm not convinced that's true. Instead, I find that waterfall development is still alive and prospering. Unfortunately, the teams leveraging waterfall methods are not prospering. In fact, I'd argue that less than half of all software development teams leverage agile practices. I have no hard data that supports my claim, only anecdotal evidence suggesting it's valid. Regardless, it's disconcerting. Why?

Given today's economic woes, the impetus is on IT to show and prove their value. Continuing with status quo will not suffice. We're being asked to do more with less - fewer resources, shorter timeframes, reduced budgets, and more. While organizations look for ways to reduce costs, IT has been given a window of opportunity. It's critical that we take advantage of it.

So what can should we do? Look at what we can do to improve the quality of the software we deliver. Bake quality into the product from the beginning instead of attempting to validate quality at the end of the lifecycle. Look at what we can do to improve the efficiency of our team. Eliminate practices that aren't directly contributing to customer value. Look at what we can do to improve the transparency of the development effort. Engage the customer and share the good and the bad surrounding the current state of the system. These are just a few of the things we need to be doing now, and each have a direct correlation to reducing cost.

Agile processes and practices can help in these, and many other areas, but only if applied correctly and pragmatically. Adopting Scrum, XP, or any other agile method won't guarantee success. It won't guarantee agility. Agility is not defined by process adoption, but by the ability of the team to deliver valued software in a timely fashion. At this years Catalyst conference, I'll be leading a workshop titled Improving the Software Development Process. In this workshop, I'll be talking about ways to increase agility and improve the development process. Surprisingly, little of the discussion will focus on agile processes. Instead, the discussion focuses on techniques to improve software quality, maximize team efficiency, increase transparency, and more. I argue this is the essence of increased agility; not process adoption. We'll discuss actionable items that teams can use immediately.

If you're attending Catalyst, and are interested in improving your software development process, I encourage you to attend the workshop. If you do not plan to attend Catalyst, but are a Burton Group client, schedule a dialog now and we can discuss the issues, how they interrelate, and what you can be doing now to improve your software development efforts.

Edwin K provides advice for building JSON RESTful services

Blogger: Anne Thomas Manes

Edwin Khodabakchian, founder of Feedly (and former founder of Collaxa), has compiled a set of best practices for building JSON-based RESTful services. He has picked up these tips while developing Feedly. Feedly is a Firefox extension for building a personal web page of your favorite feeds.

Jacob Kaplan-Moss on REST Worst Practices

Blogger: Anne Thomas Manes

Thanks to Stefan Tilkov for pointing me in this direction. Jacob Kaplan-Moss (of Django fame) has compiled some great tips on REST worst practices.

Are application silos inevitable? Maybe MODS instead.

What causes silos? Should we really care about them? These questions keep popping into my head as I have revisited my report about LINQ, ADO.NET Entity Framework (EF), and ADO.NET Data Services (Application Platform Strategies, "LINQ and ADO.NET: Brilliant and Confusing"). The subject also arises in research for an overview about an IT Governance Framework that I am writing. With my data colleagues, we have talked about Lyn Robison's MODS architecture (see the Data Management Strategies overview, "The Methodology for Overcoming Data Silos (MODS): Using the New XQuery Development Stack", and Lyn's DMS blog post, "Powerful Business Intelligence").

Microsoft's newest data access technologies foster silo building but these tools do not stand alone in this regard. IT Governance according to Peter Weill of MIT's Sloan School (Weill and Ross, "IT Governance on One Page") identifies six IT decision making archetypes, one of which encourages silo building to support rapid growth of a business. The MODS architecture admits silo busting through consistent application of XML-flavored technologies (read open standards for data exchange and access). Most of my clients complain that they have far too many silos. Most architects, including me, lament the existence of silos for hygienic reasons like privacy and security controls, data and application integration, and cost effective operations and maintenance. But are silos an inevitable creation of IT and hence unavoidable?

One cause of IT application and data silos is what I have called the "bag o' cash" (BoC) model for IT governance (the Feudal archetype according Weill). BoC most strongly yields new applications and data stores because there is a 1-1 correspondence between a business entity that wants an application and a development team that builds it - usually by yesterday. An IT organization that employs BoC will inevitably build redundant applications. Their data will be scattered across a host of DBMSes. Their applications will require copious time and effort to improve or fix. But, is this an acceptable price to pay for strong revenue growth? Perhaps.

ORM tools like EF hasten development by reducing the mental fatigue caused by design context switches between object oriented coding and relational algebra. These tools transform an object model into a relational data model with the push of a button (see the Applications Platform Strategies overview, "Object-Relational Frameworks: Finding the Right Mix"). It is reasonable to assert that button pushing takes less time and money than relational data model design and implementation. If the push-button data model works well for the application, then what is the harm in button-pushing other than an architect's disdain? If there is a well considered and properly implemented data services platform (DSP) infrastructure, then this question can be answered in the negative (i.e., the harm is to make data that should be more accessible and secure markedly less so). If not, then the answer must be something of the form - Well, it depends.

Perhaps MODS is the right answer. Build the application and its persistence store agilely but consistently. Stick with a single XML-flavor technology stack. Deploy only those commercial off-the-shelf (COTS) products that support the XML-flavor stack and are RESTful by nature. Worry integration using data services borne naturally of the consistently flavored technology stack. Employ peer-to-peer RESTful design for information exchange, thereby avoiding copious quantities of data copying. But, for many IT shops this just might not work because legacy flavored technology long ago preempted this move. And, will a business manager with a bigger BoC and a penchant for telling IT how to code and which products to buy ever be wrong?

Are application silos inevitable? It's your turn.

Patriot Act vs. EU Data Protection Directive: regulatory death-match in the Cloud

Blogger: Richard Watson

I spent part of last week speaking to European clients about cloud and SaaS.  There's a real thirst here for a conversation about the challenges and risks of cloud and SaaS, free from hype and hysteria.  The architects and executives I spoke to are already fatigued with "SaaS and cloud being pushed non-stop by vendors and analysts" (sic). 

One universal concern about hosting data in external clouds is data privacy.  Heretofore, concerns of EU companies included the fact that storing personal data in "third countries" violated the EU's Data Protection Act.    Of far more of concern now is that local data regulations in the provider's jurisdiction (especially the US Patriot Act), could be prioritized over international Safe Harbor arrangements designed to broker the local and guest privacy regulations.

I think these fears are justified and the recent clarification by the European Commission in its "Frequently asked questions relating to transfers of personal data from the EU/EEA to third countries" does little to allay that substantive fear.   The EU confirmation does indeed clarify which "third countries" have adequate standards to comply with the EU Data Protection Act, thus opening the door for EU data to be stored in (you won't need two hands to count them) the US (with Safe Harbor), Switzerland, Canada, Argentina, Jersey, Guernsey and the Isle of Man.  US companies that have signed safe harbor agreements (including Amazon, Google, Microsoft, and IBM) are bound to comply with European data protection standards, but the relative prioritization with the Patriot Act has never been explicitly tested in court.  Therein lies the problem.  Could you risk your customers' data as the test case?

This interesting study by Galexia challenges the effectiveness of the compromise of Safe Harbor agreement. 

A further grey area that was raised at our cloud sessions was whether "strategic US business interests" might also be prioritized over Safe Harbor agreements.  I pointed out that an even more concrete concern should be accidental data leakage and inappropriate use through either carelessness or malpractice of providers' employees, not to mention poor isolation practices.  These issues are well-documented in Eric Maiwald's (subscriber only) "Considerations for Risk Management When Choosing Software as a Service".

The Cybersecurity Act of 2009 recently introduced in the US Senate gives European consumers more reasons to think twice before jumping into (especially private) cloud agreements with US based providers.  The measures in the Cybersecurity Act are at the same time potentially wide-reaching and vague, but include giving the US President a lot of power over the security of the Internet.

Despite the recent EU clarification, data privacy raises key questions for European cloud adopters.  How are the clouds looking over Canada ... or Switzerland?

Gearing up for Catalyst: San Diego, July 27-31

Blogger: Anne Thomas Manes

Wow. It's April already. Less than 3 months to Catalyst.

We've been sorting through the plethora of submitted proposals, and we've just about finalized the schedule for event. I'm sorry to report, though, that two of my favorite end-user case studies had to withdraw their submissions last week. Therefore I still have 2 open slots. If you have a great story to tell about SOA or business value metrics, I am once again accepting proposals for these two tracks. (Please send me an email: amanes AT burtongroup DOT com.)

All in all, I'm very excited about the schedule. We have 3 great cases studies lined up on SOA, and 2 for business value metrics. I'm also really excited about the line-up we have planned on Cloud Computing and SaaS. It's a conference you can't afford to miss. Check out the full agenda. Also check out the workshops. I'll be reprising my REST Easy workshop.

By the way, we have a super secret discount available to readers of this blog. We have place a couple of Easter Eggs on the Catalyst Web Site. You can reveal the discount code two ways:

• Hover (but don't click) over the "San Diego" icon for 20 seconds
• Click and hold on the Catalyst logo and then drag your mouse off and release  

Open Source is not as Open as Standard Specifications

Blogger: Anne Thomas Manes

Following up on Kirk's post last week on Sun's Project Jigsaw...

The situation is actually more troubling than it appears, and it reaches beyond the question of modularity. Jigsaw is Sun's effort to incorporate a module system (JSR 294) into Java SE. After years of haggling with the OSGi Alliance, it appears that JSR 294 will support interoperability with the established and widely-supported OSGi framework (JSR 291), although it still seems pointless to me to develop two different module systems. (Note that JSR 291 is an approved spec, and JSR 294 is not.) See the OSGi blog for a bit more history on this battle.

The really troubling bit is that Sun is hard at work developing OpenJDK 7, and it is incorporating non-approved specifications like JSR 294 into it, but it has not yet submitted a JSR for Java SE 7. I guess that Sun now believes that supplying a single open source implementation of the Java SE 7 platform is good enough, and that defining a standard specification that enables other vendors to develop their own implementations is no longer necessary.

This maneuver establishes OpenJDK 7 as what Burton Group refers to as a "Rebel" framework. It's comparable to other popular open source frameworks such as Struts and Spring.In this case a very popular framework -- but non-standard, and available from only one source.

What are the implications for Java going forward if Sun decides to abandon its standardization system, the JCP? What's to become of Oracle JRockit? And IBM's implementations of Java on z/OS and other operating systems? Sun will not build ports of OpenJDK 7 for all possible platforms. Should the other vendors simply abandon their implementations in favor of Sun's implementation?

Based on the JCP Executive Committee meeting minutes, it appears that Sun is breaking away from the JCP because it doesn't want to concede to Apache's demand (which is echoed by the rest of the JCP members) for an unencumbered license to the JDK TCK. See Stephen Colebourne's blog for a summary of this battle.

All-in-all, an acquisition of Sun by IBM is sounding better and better all the time. Let's hope the deal proceeds.

Java or .Net - The Platform Dilemma

Blogger: Kirk Knoernschild

Just published Java or .NET - The Platform Dilemma . If you're a Burton Group client, you can download the overview. Otherwise, here's the abstract.

Not long ago, the decision to use Java or .NET for enterprise development was often filled with polarizing discussions.  Argumentative points advocating the merits of one platform over the other were often based on false facts and pseudo-information.  Today, many organizations have development teams that use both Java and .NET and the argument is again surfacing. The decision today, however, is based on different criteria than before.