Using Contracts to Mitigate Offshore Risks

MSA – a “horizontal” component of an offshore contract can become a powerful tool in managing an offshore engagement and mitigating its risks. My approach to turning MSA in such tool includes several main steps:

1. Identify specific risks associated with the engagement. See my earlier post Top outsourcing risks as an example.

2. Rank the risks and select top ones; limit the selection to 5-10 items. The reason I recommend limiting the list is the cost / length of negotiation process.

3. Find out the reasons the risk mitigation is not in place / insufficient. You need to understand why this presents the problem for the vendor; without that knowledge negotiations are likely to hit an impasse.

4. Identify your preferred risk mitigation plan(s). The plan should include what both parties should do to reduce / eliminate / mitigate the risk

5. Insert and negotiate corresponding language in the MSA. Keep in mind that negotiating each of the topics may require multiple revisions and some give and take on both sides. Taking a win-win approach to the negotiation from early on is essential.

Let’s consider a greatly simplified example: Let’s assume that you are negotiating an MSA with Indian outsourcing company and after second step arrived with top two risk items: “Excessive resource turnover” and “Technical capability of the resources”.

Why is excessive turnover so common? Could it be avoided? Why don’t they (the vendor) just fix it? Well, they can not. The employment situation in India when it comes to IT resource is similar to what we’ve seen in Silicon Valley during the peak of DOT COM.  Can you spell Java? Hired! Inevitably job hopping becomes common… So, facing the facts, you know that there will be turnover on the project, and it will be higher than the 20% average you vendor told you about (see my post Outsourcing Myths: Turnover Ratio).

What can you do to deal with inexorable? Here are just a few options – maintain ongoing recruiting efforts, keeping staff on stand by, continues investment in crosspollination, knowledge management, documenting everything, etc. The list of mitigating techniques goes on and on. Your vendor is probably has a bunch of them in place. Well, it’s a perfect opportunity to ask the vendor to put the money where their mouth is.

For example you can ask for guaranteed replacement of the resource in two weeks. You can consider overlap of the resources in order to perform knowledge transfer for minimum of two weeks. You can ask for periodic audits of knowledge related documentation.

An important consideration to keep in mind: some of the turnover mitigation techniques employed by the vendor do not work in your favor. The most obvious one is moving resources from project to project or client to client in order to keep the resource engaged. I would recommend consider counter measures for example if the resources are moved off your project but retained within vendor’s organization some harsher penalties / longer overlaps applied. But you do not want to push your vendor against the wall making it financially unreasonable or preventing them from doing basically a right thing.

Here is a small example of MSA language:

Vendor shall not reassign any key resource providing Services for a period of 12 months after their respective start date of providing Services without prior approval from Client, provided that Client commits to the resource ramp up outlined in Section 5 of this Agreement. Key resources shall consist of resources critical to the Statement of Work and unless otherwise agreed, will be the Project Manager, Technical Lead, Business Analyst, Architect, and Quality Assurance Lead.

Let’s now cover the technical capability of the resources. Why that could be a problem? Well, try to find good developers in Silicon Valley even today – not easy by any stretch of imagination. Your vendor faces exactly the same issues exacerbated by several factors with huge competition from multiple dimensions – multinational corps, product companies, large offshore companies, etc.

This particular issue fall’s in a category “that is a fact, it is not my problem” but if I ignore the fact it will become a problem. In any case, the quality of resources is not something I am prepared to compromise on. So what could be my mitigation techniques here?

I typically ask for direct access to resources, right to interview and approve / disapprove, etc. That is a huge issue for many vendors though, most of the vendors do not want you to handpick the resources, for obvious reasons. So, it’s likely that you would have to offset it in some way, for example ask for interviews / etc. process for named key resources and allow vendor to deal with the rest of the team. You may consider some compensation (rate, T&C). Another approach could be setting performance benchmarks and holding vendor to those.

Here is a small example of MSA language:

For Statements of Work undertaken by Vendor on a time and material basis, Vendor shall obtain Client’s approval prior to adding any resources to such Statement of Work. Client will have the option of interviewing Vendor’s resources prior to their providing Services under a Statement of Work.

Offshore Contracts Basics

In general the language you put in contracts will not change the nature of the business, will not counter the Fundamental Laws of Outsourcing, and won’t prevent things going south. Yet it is impossible to overstate the importance of a well-written contract. The goal is to develop the contract in a way that it encourages / enforces desired behaviors and provides a framework for dealing with issues, complications, and disputes. That applies to both parties – the contract has to work for you and your vendor, in that light, considering the nature of the engagement, nothing is as important when developing a contract as keeping a win-win mind.

A typical offshore contract includes two major components: a Master Service Agreement (MSA) that acts as an umbrella document covering specific terms and conditions of the engagement, and series of documents covering the specifics. Individual tasks and assignments are usually covered by documents such as Task Order, Statement of Work, Work Order, etc.

An MSA typically is negotiated once and stays in power through the life of the relationship. There are many important elements of an MSA that define the fabric of the relationship. To some degree they are vendor and even offshore agnostic. These elements fall in a “vanilla” category and typically require just basic template and a lawyer. However even these items can create a serious obstacles and require tooth and nail negotiations. Here are the main items that fall in that category:

  • Term, renewal and extension
  • Legal framework / Changes in laws and regulations
  • Security and privacy
  • Confidentiality / Audits
  • Proprietary rights / IP ownership
  • Legal responsibilities of parties
  • Indemnitification

The second group of MSA articles defines specific aspects of the relationship and should be in general agreed to prior to getting legal departments involved. One of the reasons that should be done is that there is still a long way from general to specific T&C. For example people rarely discuss the penalty for late payments before MSA is on the table. When the initial draft is presented by the vendor the number is typically 2.5% monthly which is completely ridiculous – that is ~35% on annual basis and beats some of the worst credit cards.

  • Definition of services
  • Responsibilities of parties / roles of the parties as it applies to executing the engagement
  • Payments and other financial aspects, terms and conditions
  • Initiation / Setting up ODC and other (“hidden”) fees
  • Termination

This group of the MSA articles requires very detailed analysis as it will impact Total Cost of Outsourcing in the most dramatic manner. The last article in that list, Termination, requires especial attention and a legal eye. As a buyer of offshore services you want to make sure that you can get out of the contract easily in case anything goes not the way it was planned. This is not a symmetrical clause – the vendor’s right to terminate contract should be limited to legal or financial bridge of the contract on your part.

The last group of MSA articles is not typically found in the initial draft. These are clauses that are specific to your company and nature of the engagement. Developing that list and negotiating each point is not a trivial exercise, and deserves a separate post, which I shall have shortly.

Pragmatic Outsourcing vs. Gartner

You may have seen latest Gartner view on offshore destinations via Gartner rates offshore outsourcing hot spots or variety of other sources. I find Gartner view quite interesting, informative yet sometimes not very relevant to needs of software product companies.  In this particular analysis the information is very good and applicable to a large degree with exception of a few important areas where data is misleading when applied to IT and Software Outsourcing specifically for small to mid-sized companies.

An obvious disclaimer here – my opinion is based on a fairly limited sampling – my own observations, recent experience and analysis plus some supporting data from several people in my network with relevant experience and knowledge.

If review had been based on viewpoint of IT/SW professional the rating for Labor Pool, Infrastructure, Cost & Data and IP Security and Privacy would have been different, for example Labor Pool in Uruguay when it comes to software developers is not nearly as bad as the survey presents, and as a matter of fact it is better than in Costa Rica and Mexico.  Some other ratings seem seriously off even to an uneducated eye, take for example cost of resources in Russia marked by Gartner as Very Good; I’d say they have not been in Moscow recently.

But a single rating that prompted me to make calls, talk with many people and write this article was rating for Data and IP security and Privacy in China as Poor.  Anyone who went through on-site visits with major IT Outsourcing vendors in China would tell you – there have been huge advancements in that arena, the quality of Data and IP security that the vendors could provide is very impressive.  As my friend, a VP Engineering for very successful SF startup, told me – “I’ve seen the Great Wall of China, but what really impressed me was the Great Firewall of China I saw during my trip to Beijing…”

Americas:

Americas Source Argentina Brazil Canada Chile Costa Rica Mexico Uruguay
Labor Pool Gartner F G V G F V P
Pragmatic Outsourcing F F G G F F G
Cost Gartner V G F V G V V
Pragmatic Outsourcing G G P V G G V
Data and IP Security and Privacy Gartner F F E F F V F
Pragmatic Outsourcing F G E G F G G

Europe+

Europe, Israel, S. Africa Source Czech Rep Hungary Ireland Israel N. Ireland Poland Romania Russia Slovakia S. Africa Spain Turkey Ukraine
Labor Pool Gartner G G G G P G G V F F G F F
Pragmatic Outsourcing F G F G P G G G P F G F G
Cost Gartner G G F F F G V V V F G G V
Pragmatic Outsourcing G G P F P G F F G F F G G
Data and IP Security and Privacy Gartner V G E E E G G F V V V G F
Pragmatic Outsourcing V V E E V G G G F G V G G

And Asia

Americas Source Australia China India Malaysia New Zealand Pakistan Philippines Singapore Sri Lanka Vietnam
Labor Pool Gartner G V E G F F G G F F
Pragmatic Outsourcing G V E G F G G G P G
Cost Gartner F V V G F V V F V E
Pragmatic Outsourcing P E V G P V V F E V
Data and IP Security and Privacy Gartner E P G F E P F V P P
Pragmatic Outsourcing E G V F E P F V P F

Downfall Impact: Do you keep your current provider?

Another good question posed by Michael Grebennikov in LinkedIn, when the market is down, the budgets tight and future is more uncertain than usual, what do you do with your outsourced projects? Of course this question can not be dealt with in insulation. Major market events require immediate and aggressive action, all aspects of the technology organization need to be dealt with quickly and in the most judicious manner. The organizations that do not react / change fast enough pay huge penalty. When Cash is getting low and/or P&L is looking grim organization must rationalize its R&D and Project portfolio. On my book that means spreadsheets, metrics, analysis and often concrete and resolute actions. The goal is to quickly reassess what you can still afford to keep and what must go, in all aspects of the organization: projects, initiatives, providers, and sorry to say staff.

The “to keep or not keep” question must be applied to an outsourced portion of your project portfolio. If the projects are not “keepers” there is no other question, if they are the question is whether to continue with the outsourcing… One of the way to answer that question is go back to the decision criteria you used when dealing with the “to outsource or not to” question. Reassessing the answer with the new environment in mind on a project by project basis could be one of the most reliable methodologies. It could be quite laborious though. You may consider a simplified set of criteria based on a few key dimensions:

  • Total Cost of Outsourcing (TCO) / Price performance
  • Relative Productivity
  • Quality of deliverables
  • Overhead / cost to manage relationship
  • Quality of relationship
  • Cost of termination / suspending the partnership
  • Cost of restart (with current or alternative provider)

Rating against these criteria will quickly point out suspects for termination. For remaining projects / partners you can do an in-depth what-if analysis compare status quo to an alternative approach and finally make the decision.

In some cases you will still find yourself on a fence. You are still in a grey zone if your pros / cons balance is 40/60 or narrower. In that case I would consider getting the vendor involved – assuming that your relationship with them allows. The vendor has a lot of leverage in reducing the TCO and thus pushing the odds in their favor. In ideal case – and I have been fortunate enough see those – you can work out something with the partner on a basis.

The bottom line is clear: desperate times call for desperate measures. Not recognizing it on any side of the vendor-consumer relationship is lethal for the relationship and possibly for both parties. Dealing with the issue in timely manner, looking for mutually beneficial solutions and considering “win-win” style negotiations is likely to keep both sides relatively happy.

Outsourcing Impact on Technology Choice

I find LinkedIn to be a good idea generator for blog topics, for example a question from Vinay Joshi “.Net OR Java what technology projects you outsource — Does technology matter for making decision to whether outsource or not? …” deserves substantial discussion, beyond my brief answer on the site; especially considering that the rest of answers are more about religious war of .Net vs. Java rather than about the question itself.

Of course the answer depends on the context, if you are a technology company that already has the technology selected or a vendor that has a large team with specific expertise in place the discussion has little relevance. For those about to outsource it could be quite important decision though.

If you are planning on outsourcing but have not selected the technology yet, here are a few tips to consider:

  • Flexibility offered by technology is not your friend. The more discipline the technology offers / requires the easier it is to control it, the less are the chances on-shore and off-shore teams drift apart. In particular using Java vs. .NET discussion – Java offers great flexibility and far less commonalities in solving even basic development task. There is always 10,000 ways to achieve the same objectives. It offers multiple schools of thought and competing technologies. .NET offers more disciplined approach, while it offers some flexibility it’s far less the focus or the modus operandi, typically in .NET there is “the right” way of dealing with majority of tasks.
  • Emerging technologies are not made for outsourcing. That seems like a no-brainer, yet I’ve seen many companies moving projects using cutting edge technologies offshore, typically with painful consequences. So just in case, there are many reasons not to do so: lack of experienced resources, blind spots in understanding the technology on the both sides of the ocean, insufficient supporting community and documentation, undeveloped best practices, etc. Each of these issues by itself can destroy the engagement, when the issues combined the failure is guaranteed. Both Java and .NET by themselves are established technologies, however there is always something new being pitched by the respective camp.
  • Close doors to Open Source. Well, that might be too strong of a statement. As a matter of fact I did quite well outsourcing development using Open Source technologies and products and so many people I know. Caveat emptor! If you go for Open Source make sure that you do not stray off the beaten track and stick to very stable and mature products with strong development community. Too frequent release cycle, fluctuating quality of products, unstable supporting community can add insult to injury when combined with inevitable issues of outsourcing.
  • Don’t let the tail wag the dog. Some advanced technologies come with very costly or complex development tools. Some technologies require you to invest heavily in workstations or development environment. Some technologies require extremely high investment in training. And so on. Unless you have extremely compelling reason to do so, do not consider such technologies. Investing into a partner or their environment is not what you want to do especially in the early stages of the partnership. What if the partner already has it all in place? Well, do you want to be locked into using a specific partner? I don’t think so…
  • You can only find free cheese in a mouse trap. In development today there are a plenty of “very simple” technologies. Those technologies could be quickly learned, and superficial or even spurious expertise sold to a naïve buyer. That usually attracts gazillions of providers and inevitably drives the price down. Have you heard about PHP freelancers for $4 an hour? Just go to elance.com or guru.com – you will find a plenty. The chances are you will get what you paid for. The main point here is while the technology at question could be extremely solid it doesn’t mean that any code monkey can operate it. Finding good providers in such technologies could be a challenging task due to the high pollution of the field. Unfortunately PHP today falls into that category, and I am certain tomorrow that will be the case with RoR.

Pros and Cons of Outsourcing to India

India offers the most developed, experienced and sophisticated outsourcing community. No surprise – embedded advantage of ESL, huge supply of IT talent, and low standards of living made it a top destination for IT outsourcing long time ago. Y2K and management talent solidified the success creating multi-billion dollar giants and changing ethnic landscape of many cities in the USA. As I mentioned in Offshore Vendor Selection: Choosing the Destination “if your risk tolerance is low and/or your organization is new to outsourcing go to India, you can not get fired for hiring IBM. Go to India if you have to choose on a spot, or have little knowledge of outsourcing, or have to deal with large scope ERP implementation, or … as a matter of fact if you have to ask this question chances are you should consider India as your top destination.” Now let me put a few bullets here supporting my statement:

Infrastructure. Unless your partner is tiny and located in a 3rd tier city you won’t have any problems with infrastructure. Well, you may have to deal with some irregularities in connectivity due to some natural disasters, it gets quite rainy during monsoon season out there, but I tell you that: we use AT&T as our internet provider in our San Francisco office and once in a while they drop connectivity despite blue sky and sun outside. With a huge supply of IT services in India you can find infrastructure that would cater to most ridiculous demands.

Operating Environment. Flying to India is far from fun especially from the west coast, in particular if your company doesn’t cover first class travel. 30 hours in transit plus you arrive there in the middle of the night. Unless you time your trip well the nature would great you with heat and humidity. Flying back could be so much better if you did not need to deal with airport lines and crowds. The good part, that’s pretty much the extent of the adversities. Chances are you will be staying in a good hotel, will have a personal driver, eat in good restaurants, and even corruption is wide spread in India at all levels you most like won’t need to deal with it.

Skills Availability. That’s is one of the strongest Pros of the country. No matter what skill you are looking for there will be at least 10,000 people who have it. Well, more seriously, the supply of IT talent in India is outstanding, some areas more than others of course. Mainstream technologies of today and yesterday – Java, .NET, C/C++, ERP, Cobol, etc. – have substantial oversupply. You also can find a lot of talent even on a cutting edge of the technology. The quality of the talent follows the bell curve and nowadays the median has gone up comparing to late 90th.

English Skills. Well, that’s a hidden gem isn’t it? Of course with English being widely popular in India the main issue you would need to deal with would be an accent. Maybe some idiomatic expressions, some speech forms, etc. but generally it is not an ever a showstopper and forms a huge Pro of the country.

Cultural Compatibility. While there are a plenty of cultural differences between India and USA I would put the Cultural Compatibility in a category of Pros, here are a few reasons:

  • The cultural differences on business side were not so dramatic to begin with considering history of British influence on legal and business system of India.
  • Resources from India have been in this country in large numbers and for a long time. People in the USA learned the differences, behavioral patterns, and idiosyncrasies to a pretty good degree.
  • Many Indian vendors invest a great deal into cross-cultural training as well as in accent training. As a result the gap between cultures is narrowing considerably.

There are of course cultural differences that are deeply embedded in people’s psyche, here are a few most notable:

  • “Never say No” or “Yes to Death” – while working with Indian resources you always need to keep in mind that they might have a very difficult time say “No” in any shape or form. “Can you do that? – Yes, we will do Nick.”, “Do you have access? – Yes we do Nick”. That doesn’t mean that they can cater to any need or demand, they just can’t say NO.
  • No bad news is a no-news. While the times of chopping off bad news barer heads are over, the habit is still there. So if you do not hear about bad news, it doesn’t at all mean that everything is going well, it just simply means that you do not hear / do not know what is going on.
  • Motivational hierarchy. Of course Maslow’s Pyramid rules. But there is a plenty of subtle differences in how its upper levels translate for a specific culture. Not bad / not good – just different. For example, personal success in India outsourcing is often measure in number of people the person supervises. “I have 100 people under me…” That pushes good developers away from the technical track towards managerial with inevitable profound negative impact on technical abilities of the organization.

Rates. India rates fall neither into Pro nor into Con category. They are benchmark against which other rates are compared. And I guess that makes for a nice segue into Cons discussion:

Resource Turnover. Turnover is very high, it is high to a degree that it almost outweighs all pros of the region. See my earlier post Myth for more thoughts on the subject.

Resource Quality / Technical Capability. IT Outsourcing proved to be a rather lucrative business for many social groups in India – entrepreneurs, engineers, education providers, etc. Millions of people moved into the field in the Golden Rush of the century. As a result average quality of resources started going down to a degree that even time-proven trademarks of quality do not work anymore. Not long time ago I was stunned when I had to fire a consultant for incompetence; the stunning part came from the fact that he had a master degree from IIT.

One more Con related to the Golden Rush is worth mentioning: huge number of companies with a large number of low quality fly-by-night vendors makes it extremely difficult to find a right provider. It’s very much like looking for gold – you have to go through the tons of dirt to find the right substance. However, you are looking for gold, and one thing I am certain of is that you can find that gold in India.

Don’t Fall Asleep Behind the Wheel

This post is a rude reminder to all of us involved in outsourcing. Just a few weeks ago I talked about fundamental laws of outsourcing (FLO). And yet I just escaped from being hit hard by one of them, the ominous Second FLO, by the skin of my teeth.  Quick note – the second FLO as the same as the second law of thermodynamic – entropy always increases. In offshore outsourcing the second FLO exhibits itself as consistent degradation of quality of services in absence of non-stop energy applied from the on-shore.

In this particular case it was about sourcing. I have an agreement with my current provider that I can interview any resources prior to them being assigned to the project and I can stop that assignment from happening solely on the results of the interview. I have to say that is a somewhat unusual agreement – most of the vendors would fight tooth and nail against it; that did not stop me on many of my contracts though.

Anyway, my vendor has been good in many respects, with quality of the resources being one of the top. So when I did not see any red flags on the resumes of a couple developers about to be assigned to the project I was ready to give the green light. I am not sure what stopped me and why I decided to interview them. But as soon as I asked for the interview all kinds of red flags stared coming up: scheduling delays, preparation phrases, language and cultural difference discussions… Making the long story short interviews were a complete disaster: the guys were not only exceptionally green, they did not have the foundations I was looking for, no grip on technology, no relevant background… they were good guys ready to learn. Sorry, but I do not pay for on-the job training…

So what happen? Why my trusted partner was about to put these spring chickens on my project? Well, just because. The second fundamental law of outsourcing is as strong as the gravity laws. You know, you can fight the laws of gravity as long as you want and yet you will end up with you face in the dirt… Uninspected deteriorates. [Dwight David Eisenhower]

Here is a metaphor for your consideration. A long time ago I was in the far north of Siberia, in Eskimo country. I saw an amazing event there – sleds led by sixpacks of reindeer were competing in a traditional race. That was indeed a fun race and a very vigorous exercise for the jockeys – they had to use a very long stick to control the deer, and make them run. Interesting thing about those deer – they do not run if you do not hit them, and, unlike horses, they would stop the second you stop hitting them. So the only way for the jockey to win the race is hit them non stop all the way to the finish line. Little did I know that many years later I would have to use the same technique on all my offshore engagements.