Negotiating a Fair Rate

Let’s assume that you have selected a few companies for your shortlist and are getting close to the final stage of negotiations. At that point you already have the “asking” rate, which should be within reasons. Negotiating space in offshore deals is rarely above 30% and if asking rate is 100% above your expectation the vendor should not probably be on your list.

Now, how to make sure that you get the best rate and at the same time not push your vendor beyond the line where your negotiating “success” will backfire? The key is to drive for “win-win” arrangement with every prospect vendor and pick the best one. Here are a few tips on doing that:

  • First, most important, you need to set the focus of your negotiations. Your goal is not to minimize the rate but minimize the Total Cost of Outsourcing (TOC) over the terms of the engagement. TCO is an abstract concept unless looked at in retrospect, yet it could be reasonably assessed with some basic assumptions. After the assumptions are locked you can easily negotiate towards minimizing TOC.
  • Next step is arranging your arsenal of negotiation options. You do not want it “all come down to rate”, that’s just one aspect of TOC plus it’s likely to stall the negotiations. List all aspects of the contract you could to negotiate and form your position on each of them. Offshore contracts typically offer large number of areas to negotiate, e.g. financial and payment terms, work hours, overtime rates, length of engagement, access to resources, multiple operation benchmarks and guarantees, etc. Each of these items could have a massive impact on the TCO.
  • Research what is reasonable in terms of rates for your vendors. There are plenty of tools to do that. For example follow up with references the vendor should have given you (it’s amazing how much info you get if you just ask), check regional job boards to determine salary ranges, etc. Take you research data with grain of salt though, e.g. be aware of the timing of your data. For example I was involved in contract negotiation with the same vendor in 99, ’02 and just recently; the terms of the contract were somewhat similar, the rates for mid level java developer were respectfully $42, $21 and $27 an hour.
  • Negotiation is a complex skill if not art. If negotiations are not particular your cup of tea you may consider involving professionals, in particular those who have experience negotiating offshore contracts. At least go through some serious reading on the topic prior to diving into the deal making. Here are a few great books to consider: Secrets of Power Negotiating by Roger Dawson, You Can Negotiate Anything by Herb Cohen, and Getting Past No by William Ury.
  • One aspect of the rate is often gets overlooked – the rate changes overtime. The easiest approach here could be locking rate for the term of the engagement, yet it might be not feasible due to many reasons. You want to make sure that you do not get hit with huge changes and at the same time you do not want to find your vendor loosing money on your project due to for example natural changes in the cost of living. Linking rate changes to some objective index might be a path to consider, see an example below.

X.  Cost of Living Adjustment. With respect to the rates stated in each Work Order with a term longer than one year, commencing on the first anniversary of the effective date and on each anniversary thereafter during the term of each such Work Order (each, an “Anniversary”), if the Employment Cost Index, Total Compensation, Not Seasonally Adjusted, Private Industry for Professional Specialty and Technical Occupations published by the Bureau of Labor Statistics of the United States Department of Labor (the “ECI”), as published on the most recent date the ECI was published prior to the Anniversary of the current year (the “Current ECI”). is higher than the ECI published on the most recent date the ECI was published prior to (i) the effective date of the Task Order with respect to the first Anniversary; or (ii) the Anniversary of the previous year with respect to all subsequent Anniversaries (the “Prior ECI”), then, effective as of such Anniversary, the then-current rates for such charges shall be increased by an amount calculated by multiplying then-current rates by a fraction, the numerator of which is the Current ECI and the denominator of which is the Prior ECI, minus 1. For example, on the first Anniversary of a Task Order, if the Current ECI is higher than the Prior ECI, the increase to the then-current rates under this Section 5 would be calculated as follows:

Increase to then-current rates = the then-current rate on the first Anniversary X ((Current ECI / Prior ECI) – 1).

If the ECI ceases to be published, then Consultant and Client will agree on and substitute another comparable measure published by the same or another reputable source.

Offshore Developer Rates

What is a fair rate for a mid-level Java developer working offshore? Seems like a simple question, yet the answer you are likely to receive from anyone familiar with the subject is “It depends…” A fair rate you need to negotiate towards to with your supplier depends on many attributes and circumstances. Here are the most important:

  • Location. Almost like in the real estate business location plays utmost important role in the cost of the product (rates in this case). Location granularity is roughly at a city level, meaning that in a single city you will have roughly the same rates for specific position. Large cities such as Bangalore, Beijing, and Moscow may have some pockets / districts with higher / lower rates, those differences are not as dramatic. Raising level of granularity to a country level skews the results significantly unless you limit your horizon to only “first tier” cities.
  • Other geopolitical factors. In countries experiencing explosive growth or political turmoil standards of leaving fluctuate greatly and that inevitably leads to dramatic changes in rates. Rates of vendors from Eastern Europe and China have been growing at the highest rate recently. It’s no surprise considering major improvements in standards of leaving of these countries and weakening dollar as well.
  • Competency. That’s an interesting phenomena I have observed over the years. It appears that engineering community competency has very notable local preferences. For example there is a great deal of skills in mobile development in Russia, Vietnam developers seem to prefer to speak .NET, you find many developers working with OS cores in Israel.
  • Company size. Unlike in the food industry where large chains offer lower prices s/w outsourcing has opposite trend – typically you will be able to negotiate better rates with smaller shops.
  • Vendor business model. In high-level view there are several business models which offshore organizations operate on:
  • “Body shop” – under this model the vendor is focused on billable hours / resource utilization and is typically in the business of selling mediocre resources in bulk. This not the business model you will find presented in RFP or website of the vendor, however you will see it between the lines of the proposal, in general practices, etc. This model scales well and you can see body shops ranging from Krishna’s Shack to multi-nationals of colossal proportions.
  • “Consulting Organization” – same as above but with vigorous attention to the quality of resources. These organizations are typically smaller and have much higher quality of the resources.
  • “Boutique shop” – I use this term for small sized high-end consulting firms which offer top quality resources often in a very narrow field / niche.

Rates naturally would be the lowest for first model and the highest for third. The questions of course is appropriate analysis as most of body shops present themselves as consulting organizations and some smaller one pretend to run “boutique” operations.

  • Engagement model. There are plenty of models you can elect to work with offshore, for example resource augmentation on T&M basis, fixed bid engagements, Built-Operate-Transfer, Managed ODC, etc. Each model will offer slight adjustment to the actual rates.
  • Contract details. Rate can vary greatly depending on the details of your contract, with each element being a double edged sward though. For example you can reduce the rate by committing to large number of the resources or longer term of the engagement, by agreeing with termination fees, etc.

Here are a couple links for more detailed view on this subject:

Offshore outsourcing statistics for services in 2007
Global Services Location Index (GSLI) for 2007
Choosing the Right Country for IT Offshoring

And finally, the rate table, take it with a huge grain of salt though

Group of Countries Outsourcing sweet spot Rate Guideline, USD an hour
Canada, Israel, Ireland some Eastern European countries such as Hungary, Romania R&D activities, Java/.NET, mainframe, product development 35 +
Most of Eastern European countries such as Russia, Ukraine, Czech Republic, Poland, some South American countries such as Brazil, Argentina Mobile development, some R&D, C/C++, Java/.NET, product development 25 – 45
India ERP, maintenance, mainstream development, Java / .NET, QA 20 – 35
China, Philippines Java / .NET, QA 15 – 25
Pakistan, Malaysia, Vietnam, Chile, Bolivia and many new outsourcing players TBD 10 – 20

It’s Not Over, Till It’s Over

I was fairly certain that an offshore development company with majority of their staff in St. Petersburg, Russia was the best choice for a large scale initiative for my company. The decision came after complex vendor selection process which included on-site visits, marathon interviews, long and pricy MSA negotiations, etc.

I hang up the phone after final discussion with the CEO and smiled. I liked the team in Russia, some of the guys I met there were at par with my best developers in-house, I was happy with the location as it was offering a cure for my nostalgia, and I was proud to be able to deal with the biggest obstacle I faced on the day one of the negotiations – substantially higher rates of developers in Russia comparing to those in India.

“Why work with India if you can find more expensive developers somewhere else?” is not exactly the question you want to be discussing with BoD or your executives. My convoluted negotiation scheme has paid off. I was quite happy with what I was able to squeeze out of the vendor. There were only a few formalities to take care off and I would move forward with the project.

I have t ell you – having had set my eyes on the Russian vendor I had to stick my neck out a great deal. There were plenty of concerns with outsourcing in my organization to begin with, moving it to Russia was a challenge of a much high caliber. “If the creator had a purpose in equipping us with a neck, he surely meant us to stick it out.” [Arthur Koestler] Those are the words to live by. I was selling idea of using the Russian team as there was no tomorrow. My efforts were paying off on that side as well – I had full support of my executive team and was ready to move forward with the contract and a very hefty budget.

Next day I was on my way to LA, long weekend offered a perfect break from the grueling selection process and contract negotiations. I was in a wonderful mood and almost all the way through the 10 hour trip when I got a call from my vendor’s CEO. For some reason he decided to speak in English: “Nick, my board of directors has reviewed all the details of the contract and after much discussions had made the decision to withdraw our proposal and exit the negotiations.” I said something borderline polite, hang up the phone, and issued a very loud, long and very politically incorrect tirade…

Funny enough things worked out to the best, the vendor that was awarded the contract instead of the Russian team in many respects offered a better match, stronger skills and less time difference… Plus, I added a few more notes to my bag of tricks, tips and traps:

  • Never come to the finish line of selection process with a single vendor in mind. Make sure that your short list has at least two, better three capable companies.
  • Do not rely on your intuition (bias, preferences, etc.) when selecting the vendor; let the facts, spreadsheets and team consensus drive the decision.
  • Do not oversell your team, company, and execs on the benefits of outsourcing and especially on any specific vendor. Remember no matter how low you set the expectations your offshore vendor will easily fail them.
  • Do not get lost in complex gambits and convoluted negotiation schemas.
  • And the most important, do not try to pay the vendor less than they can generally get in the open market.

The last bullet deserves special attention as a fair rate is a moving target and depends on many aspects and circumstances. I guess that will be my next post.

ODC Hidden Fees

The conference call I had with my ex-partner in Noida, India was quite unusual and it’s worth mentioning. Two PMs, AM and I were going in circles for 30 mins in attempt to solve a $10,000 conundrum. About a year ago we purchased 3 servers for our team in India, at roughly $3K a piece. The relationship came to an abrupt end due to major reshuffle of the road map on our side a few months ago.  Naturally we wanted to get our servers back in a form of iron or cash.  That turned out to be unreasonably costly.

As it turned the servers were in some special industrial zone and to get them out of the zone we would have to pay a de-bonding fee to India customs.  The fee at this point would be about 75% of the price of the new server, supposedly if I would wait for about 5-6 years the fee would be reduced to almost zero.   But today to get the servers to San Francisco between shipping and customs I would have to pay the price of new server for one year old box I already “own”.  I could not even donate the boxes to any of my friends in Noida – they would have to pay de-bonding fee to take the servers off the vendor’s premises.  Talk about hidden fees and small print!

That would be my latest lesson learned on ODC hidden fees.  There are a plenty of others you need to be on look out for when negotiating a contract with an offshore vendor.   Here are just a few I came across over the years:

  • Telecommunication, a variety of expenses associated with connectivity and other telecom needs
  • Setup costs, including workstations and other expenses
  • Administration fees, a bizarre combination of fees including janitorial, rent, etc.
  • Recruiting fees in a wide variety of contracting clauses
  • SW Licenses, sometimes for just “non-standard” components often for all
  • Termination fees, a variety of clauses preventing from or penalizing you for canceling agreement
  • Variety of financial fees, such as late payment fees with percentage higher that you would see from credit card vendors

What is important here is of course not the fairness of the fees.   You need to inspect every inch of the contract, every caveat and clause to avoid surprises and control your total cost of outsourcing.  Hopefully that will save you from serious blunders and oversights like mine.

And just FYI, thanks to Chris Balmain, here is a link for de-bonding procedures http://india.ewasteguide.info/files/Debonding_Summary-Report_2007.pdf

Offshore Risks: Team and Personal Impacts

Transferring even a small portion of your development offshore has inevitable impacts on your team and yourself. The impact could be dramatic to a degree that it defeats the purpose of outsourcing. Each of the dimensions of the impact should be considered a risk that needs mitigation plan and is dealt with efficiently through out the lifecycle of outsourcing. I’ll touch upon most significant areas:

  • Loss of team support / respect / relationships with the team. Even the most open minded employees on your team will be concerned with offshore introduction. And they should, the practice of outsourcers replacing the sheer fabric of the company, it’s all too familiar. As an instigator of the process you are likely to become a target of negativity. It comes in all shapes and forms with essence being “you are a traitor of ”. I remember well one of my key architects giving me an ultimatum “it’s me or them”. I do not know of any bullet proof shield here, the chances are some percentage of loss will happen no matter what you do. For me the best risk mitigation strategy in this case has always been transparency and honesty – when you can afford it. I do my best to personally deliver the message to every member of the team or alternatively setup a process which ensures consistent and accurate delivery of the message.
  • Loss of team spirit / internal unease. Mutual trust even in small teams has some level below 100%. Even a perfectly delivered message will be taken with a grain of salt and generate negativity. So only medicine here is reinforcement of the message – positive reassurance is like food – you can not get enough for life time in one seating.
  • Decrease in team’s productivity / commitment. Loss of key personnel / technology and business knowledge loss. Loss of team spirit / internal unease even if managed well is likely to result in tangible losses. You need to plan for them in advance of introducing the idea into your organization. Do you have sufficient redundancy in your organization to deal with inevitable loss of key personnel? Are your schedules have sufficient padding to cover for loss of productivity? Are your knowledge transfer / retention devices in place? If answer to any of these and similar questions is ‘No” you need to deal with closing the gap first and searching for vendor after that.

On a personal front the risks are substantial as well. What would championing an offshore initiative would do to your career? What’s your organization’s risk tolerance? What is its failure tolerance? How would the failures of the vendor affect your position in the organization? And so on – there are countless questions to ask here.

But even more important set of questions is around lifestyle impact. Are you prepared to shift work hours? Are you ready to deal with the never ending stress? What is your own failure tolerance?

I remember welcoming Paul Lake (an outstanding account manager for a prominent IT outsourcing company)  to a role of AM on an offshore engagement.  While no stranger to IT outsourcing he had never dealt with offshore side of the house.  “That’s the end of the life as you know it…” – I told Paul – “You will now need to learn how to start every call with “I am sorry, I have to apologize…” I wish I was at least somewhat wrong…

Offshore Vendor Selection: Site Visits

The idea of this post partially came from Offshore Visits. Tom’s post is about visits to an existing vendor, the visits during vendor selection process are quite different though.

I am fairly convinced that the unwritten rule – “must visit perspective vendor site to make a selection” was originated by offshore sales force. This is a well known closing technique similar to test driving the car (“must drive before you buy”) a.k.a. “puppy close”. The goal of the salesperson is to build your commitment to purchase and getting you involved, getting you to invest the time paves the road to closure.
From vendor’s standpoint on-site visits help tremendously in many other dimensions, for example having you on their territory (care salesperson would take you to his office “just to discuss some details”), playing one of the strongest powers of persuasion – reciprocity, etc.
Does it mean that to avoid high pressure of sale you should avoid on-site visits? Not at all. You just need to keep your eyes on the ball and counter the pressure. Shear understanding of the role the site visits play in the selection process will help a great deal. Also here a few simple techniques that should help you counter the pressure:

  • Visit several vendors on the same trip.
  • Separation of duties may help a great deal – if it’s at all possible use people not involved in the final decision making to go on the trip.
  • Be the driver of the agenda for all meetings on the trip.
  • Concentrate on your needs and things you should accomplish on the trip.
  • Take a few days off after the trip to do some personal travel and enjoy the scenery which you may never have a chance to see again.

The main goal is to make progress in your vendor selection in a way beneficial to you and not helping vendor to close the deal. In that light I find site visits extremely helpful and educational. My typical agenda would include:

  • Interview with perspective team (I usually go for marathon interviews – 20-30 people per company, 20-30 min per person)
  • Informal interview of managers, executives, etc. – I do not interview them per se, instead get a chance to see them in action
  • Review infrastructure, take a pick in server rooms, see desktop environment
  • Check out employee lifestyle – workstations, libraries, transportation, food, break rooms, wallpapers, etc.
  • Assess physical and other aspects of security

I do my best to avoid:

  • Meetings with large participation and no purpose
  • Sales presentations in any shape or form
  • Any optional activities that do not pass sniff test

Offshore Vendor Selection: Criteria

There is no one-fit-all list of vendor selection criteria, as the list depends on the organization, its needs, scope of outsourcing initiative, etc. I would like to suggest some items that you may want to consider for including in your list.  To offer some structure to the list I put items in five major groups:

  • Macro Factors. This group includes criteria important for selection of outsourcing region and type of outsourcing organizations to pursue in your search.
  • Critical Factors. The group of selection criteria covers items that are critical for the success of the outsourcing initiative.
  • Relevant Factors. While these search criteria are important I always take them with a grain of salt as assessment of these factors is complex / subjective.
  • Things to Consider. Tie breakers, items of relatively low importance which could still affect your decision with consideration of special circumstances, etc.
  • And Personal Factors. That is a very important group of criteria if you are the person who has to carry on the engagement. If you only charged with selection these items should reflect needs of future stakeholders, assuming they could be reasonably assessed.

My recommendation is that you take a critical look at the list below and pick only items of the high importance; selecting vendors based on an excessive list of criteria would be a daunting task.

Macro Factors

  • Geography – Region / Country / City. Impact of locale on outsourcing initiative is multifold; in large degree the geography determines time differences, language, culture, history, average rates, typical turn over ratio, etc. There are however some notable fluctuations based on specific city and company. For example some companies in Eastern Europe change their work hours to minimize time difference.
  • Political Stability. Political stability will affect multiple aspects of an outsourcing relationship, an impact range from minor fluctuations in productivity to mandate to cease the operations.
  • Legal System Maturity. Generally you are not planning on going to curt with the vendor yet you should never dismiss a possibility of that happening. What’s good about your NDA or MSA if its clauses are not enforceable?
  • Company Size and Organizational Structure. The attention you receive from the vendor is directly proportional to percentage of revenue your business represents; see offshore outsourcing Rule # 1.
  • Business Model Match. Search for vendors that support and have track record of executing under the outsourcing model that is right for your organization. See Vendor Selection Rule # 3.
  • Financial Stability. Multiple aspects of financial stability are important factors in the selection process, with most important being profitability and cash reserves.
  • Time in Business. The time in business should be defined as the time in doing business in specific model / under specific circumstances matching your needs. See Vendor Selection Rule # 3.
  • Business Focus. Many outsourcing vendors chase any and every business opportunity inevitably creating organizations with unstable structure, staff and culture.

Critical Factors

  • Capability Maturity. Certification level. See Vendor Selection Rule # 2. Make sure that certification is applicable to the specific ODC / location you are considering.
  • Methodology Match. Is methodology you are planning to use is in DNA of the vendor? Does vendor has a track record / history of using specific methodology in the target ODC? See Vendor Selection Rule # 3. Keep in mind that methodology is bigger than just SDLC, it should cover project / program management and other essential process.
  • Delivery Track Record. I would give that criterion a very high weight and would do my best to assess the track record of success on engagements similar to your initiative. Keep in mind specific aspects of your engagement such as change rate ratio, urgency of deadlines, etc.
  • Staff Competency & Domain Expertise.  It’s clear that staff competency is critical to the success of the initiative. To rate that criterion you would need to decipher what the competency of staff assigned to your project would be. That would require understanding vendor current state of competency, sourcing and competency building methodologies.
  • Training. Tightly related to previous item this one is all about assessing what vendor does to sustain and increase competency of its employees.
  • Knowledge Transfer and Retention. With turn around being unavoidable knowledge transfer and retention are critical to “average competency” of the organization. To rate that criterion you will need to understand what tools, processes, and methodologies are used by the vendor to assure continuity of knowledge.
  • HR Practices, Staff Sourcing & Development. Vendor activities in order to find (“source”) and retain its employees are critical to countering turnover and increasing “average” resource competency and quality.
  • Data Security & Privacy.  This criterion is particular important if your organization is dealing with sensitive data such as financial or HIPAA covered data. You need to look into multiple aspects of security (policy, physical, network, data, etc.). Assessing state Data Security & Privacy could be very time and resource consuming. Asking for results of a third party audit is probably the easiest short cut.

Relevant Factors

  • IP Protection. IP Protection is a critical factor yet assessing it is extremely challenging, see some of my thoughts in Protecting Data and IP when Outsourcing Offshore.
  • Organizational Mission.  Alignment of organizational missions may have a good positive impact on success of your project. The chances of finding the alignment are not high though.
  • Client Management. Client management or account management is especially important when access to the top is limited.
  • Quality Management. Some organizations, in particular those high on CMMI scale, offer somewhat independent quality management which could be a value ad to the engagement in a long run.

Things to Consider

  • Organizational Statistics. Organizational statistics, such as %% of sales vs. development, %% of people with advance degree, etc. could offer interesting insights on the organization and suggest trends.
  • Outsourcing Tools. Engagement tools such as communication dashboards, time tracking tools and others could offer substantial value in tracking the progress.
  • Specific Expertise. Narrow niche domain expertise or specific technical competence offered by vendor could be important if not critical factor. I find counting on that fairly risky.
  • Network Infrastructure. Many outsourcing companies today offer network infrastructure at par or even better than you may have in house. Its sufficiency needs to be verified though.
  • Telecom Infrastructure.  Similar to the item above, strongly related to geopolitical criteria.

Personal Factors

There is a great variety of personal factors that should be considered. They range from personal preferences to geography to your own risk tolerance, from your career aspirations to knowledge of foreign languages. And depending on your relative weight within organization they might be even more important than critical factors.

Offshore Vendor Selection: Basics

Nothing is as critical for the success of your outsourcing initiative as finding a right vendor. Right vendor doesn’t have to the biggest, most prominent, etc. It just needs to be the right vendor for you, somewhat similar to a partner in a personal life – it’s all about a match. But that’s where the similarity ends, chances are you won’t run into a perfect vendor at a friend’s wedding and unlike in personal life by no means when you meet the right vendor “you just know”.

In order to zero in onto a right partner in reasonable time you need to go through many steps; here is the backer’s dozen of the most important:

  1. Identify and engage selection process stakeholders.
  2. Verify executive sponsorship / commitment / support.
  3. Define and agree upon the communication plan.
  4. Define and confirm / get approved the scope and budget of the initiative.
  5. Identify and confirm the budget for the selection process.
  6. Define selection criteria.
  7. Decide on items you won’t compromise on.
  8. Define selection process details / procedures.
  9. Define success and exit criteria for the selection process.
  10. Identify the team charged with the process of selection.
  11. Identify the team charged with decision making.
  12. Go through the selection process and results analysis.
  13. Finalize vendor selection and move into next stage of the engagement.

That might look like a long list, well, let me tell you – it’s a short version. All the items on the list are mandatory and price for missing any one of them is quite hefty, mainly paid in frustration, lost cycles, ruined relationships and lost credibility.

Approach to process design (item 8 on the list above) depends in a large degree on your budget and timeline, scope of the outsourcing, and the organization’s culture. It could vary from a swift selection from a handful of companies to a multi-step RFP process performed by a third party. For example here is a high level breakdown of the process I recommend for midsized software organizations:

  1. Identify / finalize your target outsourcing model.
  2. Formalize list of selection criteria.
  3. Define weight for each of criteria. Multidimensional weigh tables could be helpful.
  4. Narrow the scope of outsourcing on several dimensions – geography, vendor size, years in business, etc.
  5. Develop an RFP around your selection criteria.
  6. Develop initial list of vendor – recipients of RFP.
  7. Issue RFP and manage Q&A and proposal process.
  8. Select an initial short list of vendors based on results of proposal analysis.
  9. Create a final short list by going through vendor interviews / proposal presentations.
  10. Go through on-site visits and zero in on 2-3 semi-finalists.

One of the ingredients critical for success in vendor selection is the search criteria list. There is no one-fit-all template I could share. Creating such a list is not overly complicated and I will cover it in separate post.

Top 10 Technology Tasks to Outsource

If you listen to an offshore vendor you will quickly learn the top 10 or 1000 tasks you should outsource. The chances are anything and everything that you do will be on that list. And that would be the list of things the vendor wants you to outsource. Take a look for example at Top 100 Projects You Can Outsource. The real question is about what is good for you, and of course it depends on your specific needs and challenges.   My first outsourcing item does not currently land itself well in offshore model so I did not include it in top 10 –  Data Center / Hosting. That covers hardware / networking equipment / etc. for range of systems – production, staging, development, etc. environments. The “degree” of outsourcing may vary greatly and depends on the maturity of your staff, you demand dynamics, etc. You may only take the space, bandwidth and power; expand it to fully managed infrastructure or even consider grids such as Amazon elastic cloud.

Taking a viewpoint of a midsized software developments company here are 10 items I found to be on the top of list of tasks to outsource in technology.

1. Security / Availability Monitoring. In my experience I found that achieving aggressive service level benchmarks while maintaining high level security and privacy is practically impossible for a small or midsized company without use of Managed Security Monitoring. The scope of outsourcing may vary greatly depending on your needs. In my current place I outsource security monitoring and use third parties for security testing including ethical hacking.

2.    Database Administration. I found outsourcing of DBA tasks, in particular related to supporting uptime of critical systems, extremely cost effective. Currently I use third party to monitor my production databases on 24×7 basis and use consulting from the same vendor on tough DBA design tasks.

3.    Black Box Testing. I found that outsourcing some of QA tasks, in particular Black Box testing gives me great flexibility and has good price performance. In my view the key is to outsource appropriate portion of testing leaving sufficient portion in-house for acceptance, cross checking, knowledge retention and many other key elements of the development process.

4.    Usability Testing. In my view Usability Testing is a perfect task to outsource due to many reasons with most important being an independent / objective feedback which is difficult to achieve with captive resources, other reasons include lack of in-house expertise and high cost of tools and infrastructure.

5.    Graphical Arts, Writing and other Creative tasks. Assuming that Creative tasks are not the core of your business you may find outsourcing of it to offer dramatic price performance. With huge supply of freelance and offshore talent you can find high quality resources at a fraction of the price of high end firms and even in-house resources.

6.    Tech Support. If you have to support your products the front line of Tech Support is one of the areas that can greatly benefit from outsourcing. Of course you have to be very careful in outsourcing any customer facing activities, and tech support is one of those arrears that got particular strong negative rap.

7.    SEO. Search Engine Optimization could be quit laborious and requires in-depth knowledge of the techniques, latest trends, and subtle differences. Some offshore shops specializing on SEO offer performance based approach to compensation.

8.    Software Maintenance and Sustenance. I found outsourcing of those “less glorious” tasks relatively meaningful, especially when I had a large legacy product to support. One of the value add items in outsourcing of maintenance task is potential morale boost for the local team. A word of caution here – as outsourcing of these tasks only pays off in a long run.

9.    Reporting. Developing of a large variety of reports for internal and external clients often could become a considerable burden on technology team. Using offshore resources to produce and support custom reports in experience paid off quite well in many cases.

10.    Technology Migration. Migration tasks could be perfect candidates for outsourcing, for example moving your database from MS SQL to Oracle, an app server from WebSphere to JBOSS, or migrating code from VB to VB .NET. The word of caution – some migration tasks could be IQ-demanding heavy lifting and require a lot of in-house efforts before they could be moved offshore.

I said earlier that these 10 items are on the top of my list. Well, in many cases they formed the list in its entirety. You may consider outsourcing all these items before you move on onto more challenging and riskier outsourcing tasks.

Selling offshore concept to your organization

Let’s assume that you are sufficiently sold on the idea of using offshore. If you are not, stop right there. You will not be able to sell your organization on any idea unless you believe it 100% yourself. There will be a plenty of people wildly against offshore and unless you are true champion of the idea it will be an uphill battle. More so there is a good chance that there will be a plenty of people supporting and even promoting the idea for all wrong reasons. Unless you know and believe in what you’ll be doing you will be stuck with delivering on wrongly set expectations against unachievable goals, and the time to update resume will come sooner than you think.

The next step is introducing offshore into your organization. Unless your organization has history of success in using offshore that will be a complex long-cycle process of selling the team on the concept itself, its benefits, and your abilities to mitigate the risks.

Your initial introduction may follow the traditional pattern, for example:

General state of the organization

  • Strong foundations
  • Budget cuts
  • Competitive pressure

Challenges / pain points

  • High attrition
  • Recruiting failures
  • Time to market pressure

Solutions considered

  • Local subcontractors
  • Internal structure changes
  • Offshore

Why offshore

  • Local subcontractors
    • Pros
    • Cons
    • Decision
  • Internal structure changes
    • Pros
    • Cons
    • Decision
  • Offshore
    • Pros
    • Cons
    • Decision

Expected returns

  • Goal 1 – SMART Success Criteria
  • Goal 2 – SMART Success Criteria
  • Goal 3 – SMART Success Criteria

Risks & Risk Mitigation

  • Risk 1 – Mitigation Plan
  • Risk 1 – Mitigation Plan
  • Risk 1 – Mitigation Plan

Control mechanism

  • Governance / Oversight
  • Metrics / Feedback
  • Exit strategy

You may have to sell to different groups and audiences such as execs, your team, and investors. – tune your presentations for each audience.

Couple words of caution:

  • Do not overuse “negative sales approach” (something you might have learned from Sandler Training). You need to get your audience to support the idea for a long time, you need them jazzed up enough to get over the hump of initial rejection and even more important through the turbulence of establishing the relationship.
  • At the same do not oversell – that’s a self death sentence. Remember that offshore is likely to fail your lowest expectations.
  • Do not delegate selling to offshore account managers and/or sales execs. They will offer you that and will gladly act on your behalf. They will probably do a decent or maybe a marvelous job and you will lose control over the situation. Most likely they will over commit and you will be the one to under deliver.

Unlike in a traditional “hunter” selling model with a sole target of closing the deal you have to prepare yourself for never ending “farmer” selling process. As inevitable issues creep up and buyers remorse overwhelms some of the stakeholders you will need to manage the expectations and continue showing benefits of the offshore … of course if you see any.

Vendor Selection in China

This post is a summary of vendor selection trip to China made for a purpose of s/w outsourcing initiative for a midsized product company. The main focus of the trip was “profiling” of the vendors that made on a short list after a rather involved RFP process.

Profiling involved in-depth interviews of employees ranging from Sr. PM to Jr. QA analysts. I had a chance to interview over 60 people, and I believe that I had a chance to work with a somewhat fair sampling. I would expect that if employees were selected for interview completely randomly I would have the same professional skills ratings but English skills ratings would be substantially lower.

The table below presents a summary of my view on the software teams I’ve seen during the trip. Professional skills are rated from 0 to 10; 0 means no knowledge of the key subjects, 10 means exact or above expectations for the position. English skills are rated from 0 to 10, 0 means no knowledge, 10 means fluent (strong accent, minor grammar mistakes, etc. acceptable).

Position Professional Skills English Skills Comments
Account Management 5-8 6-9 I did not interview AMs per se, I had a plenty of time to observe their work though. Skills / understanding of AM practices were not at all impressive. While the hospitality was truly commendable understanding of AM activities was far from what I would expect from professional AM / sales / presales team. In particular the ability to listen and concentrate on my needs versus out of the box presentations and sales pitches was not demonstrated.
Project Management. 4-6 5-8 I interviewed 2 PMs, 2 were dreadful, one good, the rest were
semi-decent but junior. Most of the PMs had almost no theoretical
knowledge and border-line acceptable hands-on skills; only one was PMP
certified, unfortunately he needed an interpreter to communicate. Real
hands-on PM experience was ranging from 2 to 6 years. Most of the PM in
US terms could probably be ranked somewhere between a Project
Coordinator and Junior PM.
Business Analysis. 3-5 3-7 Unacceptable. I interviewed at least 8 of them and the only one I
would possibly consider was a junior Indian girl. Most of them had
moderate English skills, but still far less than you would expect from a
BA. Their skills in written English were notably better but they really
straggled in spoken language; understanding them was a challenge as
well. Their domain expertise was not impressive even for the projects
they worked on. Functional skills such as ability to gather requirements
were very poor. Technical skills such as data modeling skills were
practically non-existent.
Junior Developers (“coders”). 3-8 4-8 Developers range from very bad to pretty good. Most of them offered
very poor theoretical skills and narrow and shallow practical. Need to
be hand-picked, but there is a large pool to draw from. I would expect a
hit ratio of 1 out of 4. I saw great deal of desire to succeed and
multitude of signs of superb work ethics.
Senior Developers (“architects”) 3-6 3-5 Very poor, most of them at best would qualify for mid-level
developers. English skills are notably worse than juniors. The more
senior the person is the more difficult s/he is to understand. The only
good guy I met (would rate highly in Silicon Valley) required an
interpreter.
Technical

Leads

5-6 4-7 Mediocre. Probably not self sufficient on tasks requiring dealing
with complex technical issues. They seem to be generally a combination
of a mid-level developer with a junior PM. I would say on both PM and
technical accounts they are a notch lower than I would expect in the
USA. On the other hand I saw a very strong drive / desire to succeed
which could possibly compensate to some degree for the lack of
knowledge.
Junior QA, Black Box 6-9 5-10 Testing skills ranging from good to very good. English at pretty
decent level (most of them came from English studies or had lived in
English speaking countries). Most of the QA analysts I interviewed
seemed to have a great personality to position match.
Junior QA, Automation 4-5 5-7 Very small pool, most of them were mediocre at best with very
limited exposure to tools. Typical “record and play back” skill set.
Most of them had a career path of black box tester to an automation
engineer (no development background).
Senior
QA, Automation
4-5 3-5 Bad. I saw only four of them though; both skills and language were
below mediocre.
QA Lead 7-8 5-8 I saw 9 QA leads and all were OK, nothing spectacular but very
focused, detailed oriented, well organized, etc. Good grasp on QA
process (very specific to the company’s process though). Understanding /
grasp of QA automation at very basic level.
General Management 5-10 6-10 Very strong business leaders with outstanding work ethics and
commendable drive. Mostly ex-pats / returnees from Western countries /
Hong Kong / Singapore. However some of them were not professionally
strong as they seem to be able to get the jobs on the raising wave of
outsourcing mainly due to their western credentials. For example one of
the execs I met was a Ph.D. in theoretical physics with no prior
consulting / sales / software experience, very smart guy with very
little experience / exposure / understanding…

Idiot Savant

It’s been almost two years since the story below shook up my organization yet it’s still quite fresh in my memory. This story stands out as rude reminder of how complex IP protection is and how many traps you need to consider. Even though the story is a bit old I still changed the names of the participants to protect the innocent…

Ravindra Gupta a senior java architect from a well respected local high-end consulting firm ThinkBig impressed us the first time we met him. He seemed to be exactly the person we’d been looking for a long time. With little hesitation we assigned him a challenging task in a user management and administration (UMA) space. He flew in our SF location and spent a couple weeks going through the analysis of the issue, discussions with our staff and me.

I consider UMA tasks some of the most challenging and was happy to have a very impressive guy deal with them. We spent days educating Ravi on functional aspects of the UMA, its issues, technical challenges, and numerous tricks and traps. I was very impressed with Ravi grasp and progress with it. After a few weeks in the office Ravi took off to work from home at the East Cost.

The year end frenzy took my mind of UMA and Ravi’s work. Ravi’s brief status reports and scant communications did not bother me much especially considering that he was apparently making good progress and we were expecting detailed technical review session right after the holidays. Unfortunately though the meeting never not took place, instead one of the first emails of ’07 was one from Ravi:

From: Ravindra S. Gupta [mailto:rgupta@thinkbigconsulting.com]
Sent: Saturday, January 06, 2007 1:33 PM
To: Nick Krym
Subject: U.M. Design Copyright

Hello Nick,

In the chaos of the holidays, I was fortunate to find some time to myself. This yielded an insight: I have to stop selling my copyright for the price of a copy. “R.S.G. Consulting” was therefore founded (finally) on 1/1/2007.

Consequently, the existing contract (made with ThinkBig) is no longer acceptable, and I want to proceed forward under the following general agreement:

I own the copyright to all parts and whole of the generic Access Control System (RACS) that I am creating. Medem does not pay for the development of RACS, but may buy a very favorably priced license. However, it is under no obligation to do so.

To demonstrate suitability-of-use, I will configure the RACS to support up to five (5) use-cases chosen by Medem, at no cost to Medem.

Should Medem then buy a license (still no obligation), it may buy my professional services to fully or partially configure the RACS for use with the new Medem system. All work done towards this task will belong to Medem (i.e. will bear Medem’s copyright), with exception of any modifications made to the RACS (which will bear my copyright). However, Medem may instead choose to have its own developers or other consultants configure the RACS.

The attached document describes the Pricing model, Availability, etc.

The design that’s taking shape is beautiful in its simplicity and power of expression. It will more than accommodate Medem’s needs for years to come.

Please feel free to contact me at any time.

Sincerely,

Ravi
R.S.G. Consulting

I red the email and picked up phone calling Mike, CEO of ThinkBig.

A month later, after the dust settled, legal bills were paid and separation agreement signed Mike and I were seeping coffee at our favorite spot at Montgomery and Bush. Mike was going in rounds about the fact that someone he’d been working for over ten years would do anything like that. In my mind there was just one answer, an expression that I had recently learned: “Idiot Savant”

S/W Development Outsourcing: China vs. India

A few months ago I went through a vendor selection process for a technology company in SF Bay Area. The goal was to find a vendor that would become a long-term partner / a part of a local development team. My clients were set on considering only two countries – India and China.

The size of a potential outsourcing deal was fairly small: ~15 people. That would roughly correspond to $1M on annual basis. The size of engagement was still big enough to give us a chance to pick a company from a large pull of vendors who seemed to be interested. Here are some observations based on our analysis:

Rates

In both countries we saw a plenty of companies prepared to compete on price and go very low just to get the deal. Larger / more mature companies had notably higher rates, many of them with very similar message “We are by far not the cheapest but we are the best”. Considering companies we liked the rates for India depending on position and company were roughly between $20 and $35 an hour with blended rate for our team ~$30; the rates for China were roughly between $15 and $30 an hour with blended rate for our team ~$20;

Access to Resources

Access to resources in high-tech centers of India is getting increasingly complex, finding It talent ion cities such as Bangalore is almost as complex as in San Francisco. Outsourcing companies also have to compete with subsidiaries and offshore divisions of multinational corporations such as IBM, Microsoft, Accenture, etc.

We still were surprised with how difficult and slow the sourcing process was. It seemed that finding even mainstream roles such as .NET developers or Winrunner QA guys was practically impossible.

We found that access to resources in China was not as complex. It appeared that companies in China were able to staff up for a project 3-5 times faster than India companies. In example confirmed by the references it took 2 months to build a 30 FTE team versus 9 months for the same by Indian Tier 1 InfoSys.

Resource Quality

The companies with $5M-$20M revenue range (our target based on scope of outsourcing) fall in a group of 3rd to 5th tier companies with inevitable impact on their access to resources which is exacerbated by general scarcity of IT talent. So it was not a surprise for us to see very poor quality of resources. Our average “hit rate” (number of people we would consider for “hire” vs. people presented to us by the vendors) was 1 out of 4.

Companies with the same $5M-$20M revenue range are the first/second tier companies in China with top pick in off campus hiring as well as other methods of employee sourcing. We saw that as a solid, and possibly the most important, advantage for Chinese firms. However our hit ratio was even lower than in India – 1 out of 5.5 due to serious communication / language handicap.

Concentrating only on those resources who we considered potential “hires” we saw a decent blend of theoretical and practical knowledge with some diversity in skills / background / experience in India. Our potential “hires” in China showed rather weak theoretical knowledge across the board. Their practical skills were solid yet extremely narrow; most of the “good” people we talked with were “pigeonholed”, and did not seem to mind.

Employee Turnover

Getting honest information about turnover, retention and attrition seemed practically impossible so we got the numbers from unsolicited references rather than from the vendors.

Most of the large s/w outsourcing companies in India have turnover rates exceeding 30%. Attrition is particular high in large centers such as Bangalore, Hyderabad, Mumbai, New Delhi. For small companies it is not unusual to see offshore staff turnover rates exceeding 50%.

Turnover rates in China outsourcing industry are under 25%. The companies that were selected claimed to have attrition rate about 15%. We saw low attrition rates as probably one of the greatest advantages of China over India.

Communications

There is no comparison in communication skills of consulting work force in India and China. Command of English language for majority of people we interviewed in India was far stronger than mine. In China the situation was opposite, more so the more skilled and senior the resource the lower his/her language skills.

Language in just one of many dimensions of communication. There are many skills important for bridging cultural differences and communication gaps. When it comes to dealing with USA companies Indian consultants have a huge lead on Chinese in many aspects, just to name a few:

  • Body language / facial expressions – much easier to understand and follow
  • Overall presentation skills
  • Understanding of professional lingo
  • Grasp on general rules of professional communications and office etiquette

With communications being one of the most important aspects of majority outsourcing initiatives India have a huge lead on China.

Mindset & Work Ethics

Talking with a large group of reference accounts (including unsolicited) gave us an interesting insight into mindset and work ethics of development teams in India and China. Here are a couple things that people had general agreement upon when it comes to resources from Indian vendors:

  • A mindset of typical outsourcing company is oriented towards revenue / profit and is focusing employees towards “billing hours” rather than customer satisfaction or success of the project.
  • High turnover rates and general acceptance of job hopping have devastating impact on resource’s attitude and work ethics.
  • Majority of consultants have over-inflated expectations in terms of their seniority, type of work they should be doing, and a speed of promotion.

We heard much more favorable assessment of mind set and work ethics for Chinese workforce:

  • With aggressive market share oriented drive of China s/w outsourcers employees of these companies are focused on high productivity and customer satisfaction.
  • Workdays of 10-12 hours are not at all unusual (note that billing is typically negotiated on monthly basis with 8 hour workdays).
  • There is a strong prevalence of team values over individual.

The workforce work ethics present one of great advantages of the outsourcing companies in China. Combined with lower levels of flexibility in job market (partially due to the country’s political and economical structure) the work ethic to some degree offsets luck of knowledge and experience.

Total Cost of Outsourcing

And finally – Total Cost of Outsourcing (TCO). TCO is the cost that accommodates for the communication overhead, lower productivity, and all other costs that are not reflected in the rate. In a large degree assessing the TCO requires substantial experimental data with a specific vendor. Based on a survey of the references during this engagement, my own experience and experience of similar companies in terms of the structure and the scope of outsourcing we came up with interesting numbers presented below. The numbers are presented as a percentage of savings / losses over typical full time employee rates:

Project Type India China
Small R&D projects Loss 25 – 45% Loss 30 – 70%
Large R&D projects Loss 10 – 15% Loss 10 – 15%
Small mainstream projects Loss 10% – Saving 10% Loss 15% – Saving 15%
Large mainstream projects Savings 10 – 30% Savings 15 – 45%
Large QA (black box) projects Savings 15 – 30% Savings 25 – 50%

Top 5 Rules of Offshore Vendor Selection

The key to building a list of vendors to consider in your outsourcing initiative is narrowing your search. There are only ~20,000 outsourcers in India alone – picking a good one should be just a walk in park… Jurassic Park so to say… Start with identifying geography, business model, maturity and size of target offshore vendors. Here are a couple tips (Nick’s Rules) to consider for narrowing down your search:

  • Rule # 1: Size matters. The attention you receive from the vendor is directly proportional to percentage of revenue your business represents. If a revenue stream your company generates for the vendor is a rounding error in the vendor’s AR this vendor is probably far from a perfect match. On the other hand you will have a major misfit if a minor fluctuation in your business needs send your vendor in a frenzy of on-a-spot hiring or forces them to lay off a half of the company. There is not magic %% and boundaries depends on your personal risk tolerance and many other factors. My recommendation is to set the boundaries early in the selection process and do not even consider companies that fall outside. I have to mention an important consideration though – many large outsourcing companies organize their workforce in business divisions dedicated to certain verticals or in some other manner. That creates smaller companies within larger organization, for example you could be dealing with $40M Healthcare division of Stayam rather than $1B organization. It’s not the same as dealing with $40M company and would have its own pluses and minuses. From standpoint of “attention share” it gets to be very close.
  • Rule # 2: Always ask for a Second Opinion. In today’s outsourcing world 3rd party confirmation of development process quality is called CMMI certification. Development processes maturity confirmed by high level CMMI certification is not necessarily going to make your life easy. As a matter of fact I have seen how it creates additional humps and complexities. What CCMI certification offers is reasonable guarantee of the process maturity and more important process and thus relationship predictability. You will face many issues during you outsourcing journey, handling them while working with CMMI5 certified organization will be smoother. On the other hand process heavy organizations won’t be able to turn on a dime and change the processes to fit your needs or circumstances. Therefore there is a strong correlation between importance of CMM and the size of your engagement. As a rule of thumb I would say anything beyond 60 man*month worth CMMI3, beyond to 20 man*years you may seriously consider CMMI5.
  • Rule # 3: Match is the key. Focus on the companies that have proven track record of working with clients akin to your company and on projects similar to those you plan to outsource. A supersized bodyshop with one gazillion of man*years experience in ERP implementations is not a good match for web 2.0 start up. Search for the match on every aspect of your engagement – technology, SDLC, vertical, etc. The supply of outsourcing talents is huge and chances are you will find what you need. Finding a good match has an incredibly high pay off and is worth the effort.
  • Rule # 4: Aim for the Top. Seek companies where you can get access to the top. Your chief architect’s brother is a CEO of a decent company in China? Your COO plays golf with an owner of offshore outfit in Brazil? Head of operations for an outsourcing company in Vietnam is your MBA alumnae? All those could become a good place to start. If you do not have any friends or connections – use professional networking and other means to gain access to the companies at the top level. Be aware though not to fall in the “relationship trap” and feeling obligated to do business with someone just because they are a friend or so. The friendship built on business is by far better than a business build on friendship.
  • Rule # 5: Remember the Babylon. One of the main obstacles and the reasons of failures in IT outsourcing is infamous communication barriers. If the organization has more than one employee it will have communication issues, needless to say the communication challenge could only be exacerbated by geographical, cultural and language diversity. So do not build new communication barriers if it is at all possible. Three quarters of your engineering team are Russians? Go to Russoft first. Do not have a single Mandarin speaking employee? Take China off your outsourcing radar.

Pros and Cons of doing business in China

From 30,000’ view Pros and Cons of giving your outsourcing business to China could be summarized as

Pros

  • Comparatively low rates
  • Low attrition rates
  • Large pool of talent in many areas
  • Superb work ethics of the workforce
  • Well organized / highly disciplined organizations
  • Staff’s desire to succeed (learning and becoming stronger professional rather than pure career move)
  • Flexibility of the contract arrangements

Cons

  • Poor English skills
  • Weak grasp on western communications style, wide cultural gap
  • Poor theoretical knowledge in many key areas
  • Weak technical skills in comparison to the mainstream Silicon Valley resources
  • Limited access to resources in several key areas (e.g. business analysis, architecture)

For most outsourcing initiatives that I managed and consulted on Cons outweighed the Pros. Yet I hope that sooner or later I find the right project and a team in China to match, more so I believe that the balance of Pros and Cons is changing as we speak; in particular

  • Poor English skills – Chinese government and outsourcing companies are making very significant investments in English training.
  • Weak grasp on western communications style, wide cultural gap – A large number of expatriates returning to China with their families after living in the USA and other countries is reshaping culture of Chinese outsourcing.
  • Poor theoretical knowledge in many key areas – I am not seeing notable changes there; possibly due to the fact that majority of service buyers are not concerned with that issue.
  • Weak technical skills in comparison to the mainstream Silicon Valley resources – Using an old joke as a metaphor – Chinese vendors do not need to outrun Silicon Valley they just need to outrun Indian vendors.
  • Limited access to resources in several key areas (e.g. business analysis, architecture) – Influx of expatriates and attention to the issue should eventually take care of it.
Of course as Cons are being addressed China may lose some of its competitive advantage in Pros category – attrition rates, cost, etc. That remains to be seen.

I always thought that China was destined to win, and if I ever had any doubts they were eliminated after my trip to Shenzhen. When you see what a combination of strong hand of the government and grass root entrepreneurship can do to transform a small fishing village into a megapolis in just 35 years you start to believe that there is nothing that China can not achieve.