Wednesday, July 29, 2009

Is your SOA hammer looking for a nail?

This guest post comes courtesy of ZapThink. Jason Bloomberg is managing partner at ZapThink. You can reach him here.

By Jason Bloomberg

It sounds so obvious when you get right down to it: you need to know what problem you're solving before you can solve it. Common sense tells you to start with the problem before you can find the solution to the problem. If you start with a solution without knowing what the problem is, then there's no telling if the solution you have will be the right one for the problems you're facing.

Obvious, yes, but it never ceases to amaze us at ZapThink that when it comes to service-oriented architecture (SOA) projects, time and again we run into technology teams who don't have a grasp as to what business problems they're looking to solve. Now, it might be tempting to dismiss this disconnect to "techies being techies" or some other superficial explanation, but the problem is so pervasive and so common that there must be more to it. As a result, we took a closer look at why so many SOA projects have unclear business drivers. What we found is that the underlying issue has little to do with SOA, and everything to do with the way large businesses are run.

The wrong question

This story begins with the SOA architect. Architects frequently call in ZapThink when they're stuck on some part of their SOA initiative; we're SOA fire fighters, so to speak. Frequently, the first question we get when we sit down with the architecture team is "how do I sell SOA to the business?" Well, if that's the first question they ask, they're already on the wrong foot. That's the wrong question! The correct question is "how do we best solve the business problems we're facing?"

SOA is not a panacea, after all; it only helps organizations solve certain problems typically having to do with business change in the face of IT heterogeneity. It's best to solve other problems with other approaches as appropriate, a classic example of the right tool for the job.

For this reason, ZapThink considers the SOA business case as an essential SOA artifact. Architects must have a clear picture of the business motivations for SOA, not only at the beginning of the initiative, but also as the architecture rolls out. After all, business issues evolve over time, partly because of the changing nature of business, but also because properly iterative SOA efforts solve problems during the course of the overall initiative.

Even when architects ask the right question, however, there is still frequently a disconnect between the business problems and the SOA approach. The challenge here is that the architects -- or more broadly, the entire SOA team -- are only one part of the bigger picture, especially in large organizations. In the enterprise context, how the business asks for IT capabilities in the broad sense is often at the root of the issue.

SOA Business Driver Pitfalls

Here are some real-world examples of how we've seen the issue of unclear business drivers for SOA play out in various enterprises. We've generalized a few of the details to protect the innocent (and the guilty!).
The SOA Mandate at the US Department of Defense (DoD) -- In aggregate, the DoD is perhaps the largest implementer of SOA in the world, in large part because they have an organization-wide SOA mandate. It's true they have high-level drivers for this mandate, including increased business agility and a more cost-effective approach to siloed IT resources. But those general drivers don't help much when a defense contractor must create a specific implementation.

This particular DoD project involved a contractor who delivered a perfectly functional SOA implementation to their client. The client, however, found it to be entirely unsatisfactory, and the entire multi-million dollar effort was canceled, the money down the tubes. What happened? The problem was a disconnect between high level business drivers like business agility and specific business problems the agency in question wanted to solve. The fact the client wanted to "do SOA," so the contractor "delivered SOA" to the client only made the situation worse.

The most important take-away from this fiasco is that SOA wasn't the problem. From all the information we gathered, the contractor properly designed and implemented the architecture, that is, followed the technical best practices that constitute the practice of SOA. In essence they built a perfectly functioning tool for the wrong job. Fundamentally, the client should have specified the problems they were looking to solve, instead of specifying SOA as their requirement.

SOA for the Enterprise Morass -- ZapThink recently received a SOA requirements document from a financial services client who asked us to review it and provide our recommendations. This document contained several pages of technical requirements, but when we got to the business requirements section, it was simply marked "to be determined." Naturally, we pointed out that among the risks this project faced was the lack of a clear business case.

Our recommendation was to flesh out the business case by discussing business drivers for the SOA initiative with business stakeholders. In response, the client told us that it wasn't feasible to speak with the business side. The entire SOA initiative focused on the IT organization, and the business wasn't directly involved.

We pressed them on this point, explaining how critical having clear business drivers is for the success of the SOA initiative. Why is communicating with the business such an issue for them anyway? Were they afraid of such interactions? Did they not know who the stakeholders were? Or perhaps there was no business motivation for the project at all?

The problem, as it turned out, was more subtle. They described the challenge they faced as the "enterprise morass." As happens in so many very large organizations, there are no clear communication patterns connecting business and IT. Yes, there are business stakeholders, and yes, business requirements drive IT projects in a broad sense, but there are so many players on both sides of the business/IT equation that associating individual business sponsors with specific IT projects is a complex, politically charged challenge. As a result, the SOA team can only look to the management hierarchy within IT for direction, under the assumption that at some executive level, IT speaks to the business in order to get specific drivers for various IT initiatives, including the SOA effort. Speaking directly to business sponsors, however, is off limits.

The SOA Capability Conundrum -- The chief SOA architect at this European firm was in one of our recent Licensed ZapThink Architect Bootcamps, and when we got to the Business Case exercise, he explained that in their organization, the motivation for SOA was to build out the SOA capability. His argument was as follows: if we implement SOA, deploying a range of Services that can meet a variety of business needs in the context of a flexible infrastructure that supports loose coupling and reusability, then we'll be well-positioned for any requirements the business might throw at us in the future.

The reasoning behind this argument makes sense, at least on a superficial level. Let's build several Business Services that represent a broad range of high-value IT capabilities in such a way that when the business comes to us with requirements, we're bound to be able to meet those requirements with the Service capabilities we've deployed ahead of time. The business is bound to be ecstatic that we planned ahead like this, anticipating their needs before they gave us specific requirements!

While this organization might very well get lucky and find that they built cost-effective capabilities that the business will need, taking this "fire-ready-aim" approach to SOA dramatically increases the risks of the initiative. After all, it's difficult enough to build reusable Services when you have clear initial requirements for those Services. Building such Services in the absence of any specific requirements is just asking for trouble.
The ZapThink Take

If you take a close look at the three scenarios above, you'll notice that the stories don't really have to be about SOA at all. You could take SOA out of the equation and replace it with any other IT effort aimed at tackling enterprise-level issues and you might still have the same pitfalls. Master data management, customer relationship management, or business process management are all examples of cross-organizational initiatives that might succumb to the same sorts of disconnects between business and IT.

At the root of all of these issues is the dreaded phrase "business/IT alignment." It seems that the larger the organization, the more difficult it is to align IT capabilities with business drivers. Sometimes the problem is that the business asks for a particular solution without understanding the problem (like the DoD's SOA mandate), or perhaps a combination of politics and communication issues interfere with business/IT alignment (the enterprise morass), or in other cases IT jumps the gun and delivers what it thinks the business will want (the capability conundrum). In none of these instances is the problem specific to SOA.

SOA, however, can potentially be part of the solution. As ZapThink has written about before, the larger trend of which SOA is a part is a movement toward formalizing the relationship between business and IT with flexible abstractions, including Business Services, Cloud capabilities, and more. If you confuse this broader trend with some combination of technologies, however, you're falling for the straw man that gave rise to the SOA is dead meme. On the other hand, if you take an architectural approach to aligning business drivers with IT capabilities that moves toward this new world of formalized abstraction, then you are taking your first steps on the long road to true, enterprise-level business/IT alignment.

This guest post comes courtesy of ZapThink. Jason Bloomberg is managing partner at ZapThink. You can reach him here.


SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

Thursday, July 23, 2009

Cloud security depends on the human element

This BriefingsDirect guest post comes to you courtesy of Andras Robert Szakal, director of software architecture for the U.S. Federal Software Group at IBM and a member of The Open Group board of directors. He can be contacted here.

By Andras Robert Szakal

I’m spending a considerable amount of time these days thinking about cyber security. Not just contemplating the needs of my customers but working with other industry thought leaders within IBM and externally on how to address this convoluted, politically charged, complex and possibly lucrative domain. Much of our thinking has centered around new government programs and game changing technologies that seek to solve the increasing cyber security challenge.

Yesterday another cyber security report was released and instead of new technology it suggested the problem was with us – the people – as in us humans. The study suggests that we need more educated smart folks to thwart those evil hackers and prevent nation state attacks. Oh, and embedded in the report is the notion that technical folks would like to have some runway in their careers. Gee that’s a novel idea that we have been pushing as part of the professional certification programs within The Open Group over the last five years. [Disclosure: The Open Group is a sponsor of BiefingsDirect podcasts.]

I’m sitting in the cloud computing stream today at the 23rd Open Group Enterprise Architecture Practitioners Conference in Toronto. Not surprisingly every presenter and panel eventually settles on the subject of cloud resiliency and securability and the need for skilled architects capable of effectively applying these technologies.

I can’t help but think that most of the industry’s challenges with implementing shared services and effectively protecting the infrastructure is that organizations don’t have the proper architectural skills. Most vulnerabilities are seeded in bad architectural decisions. Likewise,

Some headway has been made in the integration between IT and the business. But for the most part they still exist as separate entities.

many of the poorly constructed services, SOA, Cloud or otherwise are the result of poor design and a lack of architectural skill. The challenge here is that high value architects are difficult to grow and often more difficult to retain.

Back to the cyber report released July 22.

The fact is that most organizations have created an artificial barrier between IT professionals and business professionals. The line of business professionals, management and executives are more valued than the techies running the IT shop. Some headway has been made in the integration between IT and the business. But for the most part they still exist as separate entities. No wonder the cyber report suggests that prospective high valued cyber security specialists and architects don’t see a future in a cyber security career.

How do we address this challenge?

Let me offer a few ideas. First, ensure these folks have architectural as well as cyber security skills. This will allow them to think in the context of the business and find opportunity to move from IT to a line of business position as their careers grow. Ultimately, the IT teams must be integrated into the business itself. As the report suggests it’s necessary to establish a career path for technologies but more importantly technical career paths must be integrated into the overall career framework of the business.

This BriefingsDirect guest post comes to you courtesy of Andras Robert Szakal, director of software architecture for the U.S. Federal Software Group at IBM and a member of The Open Group board of directors. He can be contacted here.

Tuesday, July 21, 2009

Enterprises seek better ways to discover, manage and master their information explosion headaches

Listen to the podcast. Download the podcast. Find it on iTunes/iPod and Podcast.com.

Read a full transcript. Download the transcript. Learn more. Sponsor: Hewlett-Packard.

Join a free HP Solutions Virtual Event on July 28 on four main IT themes. Learn more. Register.

Businesses of all stripes need better means of access, governance, and data lifecycle best practices, given the vast ocean of new information coming from many different directions.

By getting a better handle on information explosion, enterprises can gain clarity in understanding what is really going on within the businesses, and, especially these days, across dynamic market environments.

The immediate solution approach requires capturing, storing, managing, finding, and using information better. We’ve all seen a precipitous drop in the cost of storage and a dramatic rise in the incidents of data from all kinds of devices and across more kinds of business processes, from sensors to social media.

To help better understand how to best manage and leverage information, even as it’s exploding around us, I recently spoke with Suzanne Prince, worldwide director of information solutions marketing at Hewlett-Packard (HP).

Here are some excerpts:
Prince: We’re noticing major shifts going on in the business environment, which are partially driven by the economy, but they were already happening anyway.

We’re moving more into the collaboration age, with flatter organizations. And the way is information is consumed is changing rapidly. We live in the always-on age, and we all expect and want instant access, instant gratification for whatever we want. It’s just compounding the problems.

We did a survey in February of this year in several countries around the world. It was both for IT and line-of-business decision makers. The top business priority for those people that we talked to, way and above everything else, was having the right information at the right time, when needed. It was above reducing operating costs, and even above reducing IT costs. So what it’s telling us is how business managers see this need for information as business critical.

[Yet] you often hear people saying that information is life -- it’s the lifeblood of an organization. But, in reality, that analogy breaks down pretty quickly, because it does not run smoothly through veins. It’s sitting in little pockets everywhere, whether it’s the paper files ... that get lost, on your or my memory sticks, on our laptops, or in the data center.

The growth in unstructured content is double the growth that’s going on in the structured world. ... For the longest time now, IT has really focused on the structure side of data, stuff that’s in databases. But, with the growth of content that was just mentioned -- whether it's videos, Twitter tweets, or whatever -- we’re seeing a massive uptick in the problems around content storage.

The whole category of information governance really comes into play when you start talking about cloud computing, because we’ve already talked about the fact that we’ve got disparate sources, pockets of information throughout an organization. That’s already there now. Now, you open it up with cloud and you’ve got even more.

There are quality issues, security issues, and data integration issues, because you

In reality, the work product of IT is information. It’s not applications. Applications are what move it around, but, at the end of the day, information is what is produced for the business by IT.

most likely want to pull information from your cloud applications or services and integrate that within something like a customer relationship management (CRM) system to be able to pull business intelligence (BI) out.

You also need to have a governance plan that brings together business and IT. This is not just an IT problem, it’s a business problem and all parties need to be at the table.
Another area to look at is content repository consolidation, or data mart consolidation. I’m talking about consolidating the content and data stores.

You really need to look at deleting what I would call "nuisance information," so that you’re not storing things you don’t need to. In other words, if I’m emailing you to see if you’d like to come have a cup of coffee, that doesn’t need to be stored. So, optimizing storage and optimizing your data center infrastructure.

People are now looking at information as the issue. Before they would look at the applications as the issue. Now, there's the realization that, when we talk about IT, there is an "I" there that says "Information." In reality, the work product of IT is information. It’s not applications. Applications are what move it around, but, at the end of the day, information is what is produced for the business by IT.

In HP Labs, we have eight major focus areas, and I would categorize six of them as being focused on information -- the next set of technology challenges. It ranges all the way from content transformation, which is the complete convergence of the physical and digital information, to having intelligent information infrastructure. So, it’s the whole gamut. But, six out of eight of our key projects are all based on information, information processing, and information management.
Listen to the podcast. Download the podcast. Find it on iTunes/iPod and Podcast.com.

Read a full transcript. Download the transcript. Learn more. Sponsor: Hewlett-Packard.

Join a free HP Solutions Virtual Event on July 28 on four main IT themes. Learn more. Register.

TIBCO joins governance ecology support with HP's SOA lifecycle software

TIBCO Software is expanding its governance solutions for service-oriented architecture (SOA) and will now provide support for Hewlett-Packard (HP) SOA Systinet lifecycle governance software.

ActiveMatrix Policy Manager and Service Performance Manager from TIBCO combined with HP SOA Systinet are designed to reduce risk and improve management of SOA environments, including identifying and defining appropriate services, managing the lifecycle of service assets, and measuring effectiveness. [Disclosure: TIBCO and HP are sponsors of BriefingsDirect podcasts.]

Governance has proved important when adopting SOA solutions by preventing delays in software delivery from compliance interoperability, security risks, and poor service quality. The topic has been a hot one at this week's Open Group conference in Toronto.

Companies that have been early adopters of end-to-end governance have seen significant results. One global telcom company saw a 327 percent return on investment (ROI) over three years, something they attributed to well-managed SOA governance, according to TIBCO.

We should expect to see more governance ecology cooperation like this one. And these same vendors should support standards and collaboration efforts like the Jericho Forum and Cloud Security Alliance, if the promise of cloud computing is to be realized. Vendors and/or cloud providers that try to provide it all their own way will only delay or sabotage the benefits that cloud can provide.

Open Group conference shows how security standards and governance hold keys to enterprise cloud adoption

This BriefingsDirect guest post comes courtesy of Jim Hietala, vice president of security, The Open Group. You can reach him here.

By Jim Hietala

Spending the early part of this week in The Open Group Security Forum meetings, I have been struck by the commonality of governance, risk, compliance, and audit issues between physical IT infrastructure today, and virtual and cloud environments in the (very) near future. Issues such as:
  • Moving away from manual compliance processes, toward automated test, measurement, and reporting on compliance status for large IT infrastructure. When you are talking about physical infrastructure, manual compliance is difficult, expensive in labor cost, and sub-optimal -- given that many organizations choose to sample just a few representative systems for compliance, rather than actually testing the entire environment. When you are talking about virtual environments and cloud services, manual compliance processes just won’t work, automation will be key.

  • Incompatible log formats output by physical devices continues to be a problem for the industry that manifests itself in problems for security information and event management systems, log management systems, and auditors. Ditto for virtual and cloud environments, at much larger scale.

  • Managing security configurations across physical versus virtual and cloud environments provides similar challenges. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]
Emerging-standards work from the Security Forum, which was originally conceived as solutions for some of these issues in traditional IT environments (in house, physical servers), will have important applications in cloud and virtualization scenarios. In fact, with the scale and agility provided by these environments, it is hard to think about adequately addressing audit and compliance concerns without standards that provide for “scalable automation.”

The Automated Compliance Expert Markup Language standards initiative will address issues of security configuration and compliance alerting and reporting across physical, virtual, and cloud environments. The revised XDAS standard from The Open Group will address audit incompatibility issues. Both of these standards efforts are work-in-progress at the present time, and our standards process is truly and open one. If your organization is a customer organization grappling with these issues, or a vendor whose product might benefit from implementing these standards, we invite you to learn more.

This BriefingsDirect guest post comes courtesy of Jim Hietala, vice president of security, The Open Group. You can reach him here.