Pragmatic Outsourcing

Tips, tricks and traps of IT offshore outsourcing

Pros & Cons of Outsourcing to Latin America

Latin America offers one of the best options for nearshore outsourcing for the USA and Canada. It also offers great resources for Spanish localization projects getting increasing important in the states. Latin America offers a large spectrum of options roughly corresponding to the countries in the region covering almost entire alphabet from Argentina to Venezuela. Each country has its own specifics and its own set of Pros and Cons; the differences between countries could be dramatic, compare for example political climate in Mexico and Venezuela. There are still enough commonalities to considering covering of Pros and Cons for the entire region.

Infrastructure. The quality of infrastructure varies greatly from country to country but it’s quickly catching up to the required standards across the region. In countries leading of LA outsourcing – Argentina, Brazil and Mexico the infrastructure is likely to meet or even exceed your expectations, but even in countries far behind the quality is still acceptable. I was surprised how solid the infrastructure was in Chile or some cities in Bolivia. One thing is extremely important – the high quality infrastructure could be found mainly and sometime only in industrial areas of these countries. It is not at all as pervasive as it is in the USA. You must validate infrastructure sufficiency before moving forward with the vendor. The simplest way to do it at superficial level is to request a video interview over Skype.

Operating Environment. Flying to Sao Pao and getting to your vendors’ HQ is Campinas takes a while but small time difference makes it much easier to deal with than while travel to China or India. Working in many of the LA countries would not be extremely complex, challenging or dangerous. You have to know where to go and where not to but chances are you will be safe and can get your job done. Chances are your vendor operates from some of the country’s most industrially advanced area with decent standards of living and acceptable infrastructure. Of course language and cultural differences can create some challenges typically easily dealt with considering general hospitality of the region and with a little help from the vendor. All that ease falls of the cliff as you step out beyond the borders of the industrial areas. Destitution of the rural areas for most of LA countries is truly disturbing; my honest advice – leave these areas to National Geographic and Peace Core…

Skills Availability. Skill availability for high tech occupations depends on specific country / city. In general it could be characterized as medium to low. Generally you can build a small team of Java or .NET developers in somewhat reasonable timeframe, but size of the talent pool is microscopic when compared with India and China. The quality of the pool helps to some degree make up for its size, but only to some degree. In my experience putting together an 8 member team of high quality Java developers / QA engineers took over 4 months. I have to say that my quality requirements were very high and I was looking for somewhat unusual set of specifics. When you are after more run of the mill skill set you probably would have easier time. Legacy technology skills and enterprise applications skills are even less common.

English Skills. Not as good as you’d expect… and why would you even expect? I worked with many countries in the region and in each of them I met engineers who spoke English better than I, but in general you have to be prepared for language barriers or for substantially impaired hiring if you make fluent language skills a mandatory requirement. Written communications appear to be in better shape across the board; however, they still cause a drop in productivity for many of the team members.

Cultural Compatibility. I find cultural differences with LA workforce some of the easiest to bridge. That could be me personally as there are a plenty of differences to be notes. A few most important areas that I have observed while working on technology projects:

  • Developers on LA teams took very long time before they could to offer their opinion or disagree with USA team members.
  • Facts and technical quality of solutions carried less weight with LA team when it came to conflict with personnel influence. For example a less efficient solution was accepted just because it had a lot of hours invested by the team members. To appease someone / protect their feelings was enough of a motivation in making core technical decisions.
  • A very high emphasis on theory and academic values versus pragmatic business decisions.

Take a look at Cross-Cultural Communication Between Latin American and U.S. Managers for a good list of the most significant differences.

Rates. LA offers great variety of rates that depends on the country from relatively high in Argentina and Brazil to moderate in Chile, Bolivia, and Uruguay. While the rates by themselves tend to be on a high side they are fairly attractive when taken into consideration with “the entire package” that includes short / no time difference.

Resource Turnover. LA countries offer better turnover rates than many of other regions. At the same time turnover on some of my / my friends’ projects outsourced to LA showed fairly high ratio. That seemed to be related more to a specific company rather than the region.

Resource Quality / Technical Capability. Quality of resources varies greatly from a country to country, from a city to city, and specifically from a provider to provider. However, in general technical capabilities of the resources are quite impressive / above average. I was able to find people with in-depth understanding of cutting edge technologies and with proven experience of working with fairly recent methodologies in many countries across the region. There is also no shortage of resources when it comes to mainstream skills such as Java / .NET / C/C++. Legacy technology skills and enterprise applications skills are less common though.

Of course one of the most significant Pros of the region it’s nearshore advantage, mostly linked to little / no time difference. The impact of it is difficult to overestimate – small time difference, similar holiday structure and bridgeable language differences makes working with teams in LA a great experience.

November 11, 2008 Posted by Nick Krym | Offshore Vendor Selection | , , | No Comments

Pros and Cons of Outsourcing to India

India offers the most developed, experienced and sophisticated outsourcing community. No surprise – embedded advantage of ESL, huge supply of IT talent, and low standards of living made it a top destination for IT outsourcing long time ago. Y2K and management talent solidified the success creating multi-billion dollar giants and changing ethnic landscape of many cities in the USA. As I mentioned in Offshore Vendor Selection: Choosing the Destination “if your risk tolerance is low and/or your organization is new to outsourcing go to India, you can not get fired for hiring IBM. Go to India if you have to choose on a spot, or have little knowledge of outsourcing, or have to deal with large scope ERP implementation, or … as a matter of fact if you have to ask this question chances are you should consider India as your top destination.” Now let me put a few bullets here supporting my statement:

Infrastructure. Unless your partner is tiny and located in a 3rd tier city you won’t have any problems with infrastructure. Well, you may have to deal with some irregularities in connectivity due to some natural disasters, it gets quite rainy during monsoon season out there, but I tell you that: we use AT&T as our internet provider in our San Francisco office and once in a while they drop connectivity despite blue sky and sun outside. With a huge supply of IT services in India you can find infrastructure that would cater to most ridiculous demands.

Operating Environment. Flying to India is far from fun especially from the west coast, in particular if your company doesn’t cover first class travel. 30 hours in transit plus you arrive there in the middle of the night. Unless you time your trip well the nature would great you with heat and humidity. Flying back could be so much better if you did not need to deal with airport lines and crowds. The good part, that’s pretty much the extent of the adversities. Chances are you will be staying in a good hotel, will have a personal driver, eat in good restaurants, and even corruption is wide spread in India at all levels you most like won’t need to deal with it.

Skills Availability. That’s is one of the strongest Pros of the country. No matter what skill you are looking for there will be at least 10,000 people who have it. Well, more seriously, the supply of IT talent in India is outstanding, some areas more than others of course. Mainstream technologies of today and yesterday – Java, .NET, C/C++, ERP, Cobol, etc. – have substantial oversupply. You also can find a lot of talent even on a cutting edge of the technology. The quality of the talent follows the bell curve and nowadays the median has gone up comparing to late 90th.

English Skills. Well, that’s a hidden gem isn’t it? Of course with English being widely popular in India the main issue you would need to deal with would be an accent. Maybe some idiomatic expressions, some speech forms, etc. but generally it is not an ever a showstopper and forms a huge Pro of the country.

Cultural Compatibility. While there are a plenty of cultural differences between India and USA I would put the Cultural Compatibility in a category of Pros, here are a few reasons:

  • The cultural differences on business side were not so dramatic to begin with considering history of British influence on legal and business system of India.
  • Resources from India have been in this country in large numbers and for a long time. People in the USA learned the differences, behavioral patterns, and idiosyncrasies to a pretty good degree.
  • Many Indian vendors invest a great deal into cross-cultural training as well as in accent training. As a result the gap between cultures is narrowing considerably.

There are of course cultural differences that are deeply embedded in people’s psyche, here are a few most notable:

  • “Never say No” or “Yes to Death” – while working with Indian resources you always need to keep in mind that they might have a very difficult time say “No” in any shape or form. “Can you do that? – Yes, we will do Nick.”, “Do you have access? – Yes we do Nick”. That doesn’t mean that they can cater to any need or demand, they just can’t say NO.
  • No bad news is a no-news. While the times of chopping off bad news barer heads are over, the habit is still there. So if you do not hear about bad news, it doesn’t at all mean that everything is going well, it just simply means that you do not hear / do not know what is going on.
  • Motivational hierarchy. Of course Maslow’s Pyramid rules. But there is a plenty of subtle differences in how its upper levels translate for a specific culture. Not bad / not good – just different. For example, personal success in India outsourcing is often measure in number of people the person supervises. “I have 100 people under me…” That pushes good developers away from the technical track towards managerial with inevitable profound negative impact on technical abilities of the organization.

Rates. India rates fall neither into Pro nor into Con category. They are benchmark against which other rates are compared. And I guess that makes for a nice segue into Cons discussion:

Resource Turnover. Turnover is very high, it is high to a degree that it almost outweighs all pros of the region. See my earlier post Myth for more thoughts on the subject.

Resource Quality / Technical Capability. IT Outsourcing proved to be a rather lucrative business for many social groups in India – entrepreneurs, engineers, education providers, etc. Millions of people moved into the field in the Golden Rush of the century. As a result average quality of resources started going down to a degree that even time-proven trademarks of quality do not work anymore. Not long time ago I was stunned when I had to fire a consultant for incompetence; the stunning part came from the fact that he had a master degree from IIT.

One more Con related to the Golden Rush is worth mentioning: huge number of companies with a large number of low quality fly-by-night vendors makes it extremely difficult to find a right provider. It’s very much like looking for gold – you have to go through the tons of dirt to find the right substance. However, you are looking for gold, and one thing I am certain of is that you can find that gold in India.

October 19, 2008 Posted by Nick Krym | Making Offshore Decision, Offshore Vendor Selection | , , , , | 1 Comment

Pros and Cons of Outsourcing QA

I saw a question on LinkedIn early this morning on a topic I was planning to cover “Pros and Cons of QA Outsourcing” – I jumped to the answer and typed up an answer while on BART, ironically it turned out to be too big for LinkedIn and so I decided to put it here. The answer is not complete and I am planning to come back to it some time. In meanwhile here are my thoughts:

1) Outsourcing QA is often a meaningful thing to do, an easy way to start and a potentially very dangerous trap. In particular companies that shed internal QA resources and move their QA operation abroad typically pay the price in knowledge loss and ultimately in degradation of the product quality. I have seen it on numerous occasions. QA engineers in well run teams often have better product knowledge than any other part of the organization, and offshoring that knowledge falls in a category of “outsourcing crown jewels”.

2) Companies in US often consider outsourcing QA for all wrong reasons, like for example “Cost”.  Cost advantage is just a myth, see my earlier post for details Outsourcing Myths: cost advantage. And going offshore for wrong reasons is guaranteed to give your wrong results.

3) QA as a subset of IT field offers a few interesting dynamics

a. It’s one of the most important areas of SDLC which typically is given the least attention.

b. The average quality of talent pool is dreadful, independently of geography. There are many reasons for that, for example here in SF Bay Area one of the main reasons is huge pollution of the pool by fraud and mediocrity – I met 100s of people who fake their QA background or think that a couple months of homegrown education makes them top notch professionals.

c. A perception of QA skills/occupation as a substandard one. It takes as much IQ to become a solid QA engineer as a Java developer, maybe more, but what can you do about perception? As one the best QA guys I’ve eve met put it talking about his resume “going to QA is like going to morgue – there is no way back”. And why don’t developers enjoy testing? Because it’s hard, it takes serious effort to put together a decent test harness, to organize your code, etc. Yet look at the average salaries, at layoff patterns, offshore dynamics – the trends are obvious.

4) Good QA engineers, automation specialists, functional testers, etc. like any other good resources are not easy to find, considering the quality of the pool, much harder. So it’s no surprise that that many organizations after interviewing a couple dozens of key-punchers who can’t tell the difference between priority and severity and ask for $90K move to offshore. The problem with such decision is obvious – it is as difficult to find good testers in Bangalore, Kiev or Beijing as it is in Boston. Of course a large supply of IT talent in countries like India and China makes it so much easier, yet the vendors in these countries have a plenty of own issues and challenges. Outsourcing the problem will not eliminate the problem, it only passes it to a different organization which is hopefully is better equipped to deal with it… But is it? Is your perspective vendor better equipped to do your job? How do you know it? By their brochures? Having interviewed hundreds of engineers in Asia, Eastern Europe, and Latin America I can tell you with certainty – far not ever vendor is equipped to do so, many of them are squarely in a business of selling mediocrity in bulk, and the quality of the resources is far less impressive comparing to what you can extrapolate based on what you see locally.

5) Having said all that I have to state that I still outsource a lot of my development and QA activities and get great results from my partners. There are a few ingredients to that success story, here are the most important:

a. Rigorous vendor selection process with the focus on “the match” between my organization and vendors’. Search for the match on multiple dimensions.

b. Resource augmentation and joined teams rather than complete outsourcing.

c. Abundant communications in all forms with fair portion of face-to-face meetings and on-site / offshore swaps

d. Control and ongoing preventive maintenance in all aspects of the engagement.

e. Adjusting SDLC to accommodate for idiosyncrasies introduced by offshore.

f. A “disposal outsourcing” model that I worked out with my one my partners – augmentsoft panned out quite well in QA arena.

g. Working with nearshore partners was much easier in many aspects especially when running agile projects.

October 15, 2008 Posted by Nick Krym | Making Offshore Decision | , , , | 2 Comments

Offshore Destinations: Russia

I was born in Moscow, USSR and the word “Russia” in my mind associates with a large empire of 15 republics. Things since than have changed dramatically and referring to some of the parts of ex-USSR as Russia is not just politically incorrect. Yet you are likely to hear about Russian outsourcing even if the ODC is located in Minsk, or Kiev. As a matter of fact in many respects outsourcing landscape of Byelorussia, Ukraine, and Russia has a lot in common. More so, large outsourcing organization such as ePAM, Luxsoft, and others have offices in these countries. Most of other countries of ex Soviet Union do not play significant role in offshore market, some due to low density of IT talent, some due to high cost. While offering in these countries exist, and you may find great providers in Estonia, Moldova, and others, in terms of outsourcing statistics these countries would be a rounding error. With that in mind let me cover some Pros and Cons of doing business in Russia.

  • Infrastructure. IT infrastructure in large cities of Russia is very good; smaller, second tier cities lag behind, the difference if pretty dramatic. Generally today you will find sufficient network bandwidth, stable connectivity, and solid pool of Sys Admin talent that would allow you stay in touch with your ODC. The cost of it will be not inconsequential though and needs to be taken into consideration. A very important aspect of infrastructure which you need to asses is vendor facilities – it is difficult to find well equipped offices with quality server rooms, etc. that is especially serious for companies with offices in second tier cities. In my view Pros here outweigh the Cons.
  • Operating Environment. Running offshore engagement with Russian ODC will offer many operating challenges even if you stick to tier one cities (for the purpose of this discussion that’s Moscow, St. Petersburg, Kiev, Minsk). Getting to these cities is fairly easy, they offer great selection of hotels, solid municipal infrastructure, and … mind boggling prices. As I heard Moscow has been recently awarded with a title of the most expensive city in the world with St. Petersburg following it closely. Second tier cities are substantially cheaper but you get what you paid for in terms of quality of hotels, food, transportation, etc. Another issue to be aware of is high crime rate (accidental traveler be aware!) and very high rate of corruption. Corruption could become a very serious obstacle for models involving ownership of the resources such as BOT. With caveats considered I would still put Operating Environment as a Pro of doing business with Russia.
  • Skills Availability. That is in my view is one of the weakest traits of the region. First at a very high level, Russia produces IT resources at a fraction of speed of the countries such as China and India. This problem is exacerbated by fairly consistent internal demand for IT resources and high geographical dispersal of the talent pool. In large degree Russia talent pull is already exhausted. Pretty much everyone who is interested in working in offshore organization is already working for some client, often for several, as many of talented engineers work several jobs, moonlight or find other ways of get themselves reasonably compensated. Finding software aces is challenging even in second-tier cities, in the first tier cities it’s practically impossible.
  • Cultural Compatibility. My experience in that arena has been surprising to say the least. I left Russia in ‘91 as an accomplished technology professional with almost 10 years of experience under my belt. I had not expected to have any problems in dealing with companies in Russia and yet I found it easier to work with companies in India instead. Some of my greatest pains came from several areas of communication / work related behaviors.
  • Customer is always right… Maybe, but not in Russia. As a matter of fact the vendor seems to always know what I want better than I do.
  • Being “Politically Correct” is not a Russian way. However, while I prefer straight forward communications I do not enjoy when my vendor is rude to me or more so to some of my employees.
  • Work ethics. Very sensitive topic, I have seen many great, hardworking developers in Russia, but unfortunately they seem to be outnumbered by short-timers with “get money and run” attitude.
  • There is one interesting aspect of Russia’s culture which while “positive” contributes to the difference – attitude towards education. It is amazing how many highly educated people you find among Russian developers and even QA engineers. I am not talking BS, I mean Ph.D. and above. While by all means commendable quality the negative impact of it is actually multifold: theoretical approach to problem solving, abandoning career for the sake of education, investment in education at cost of work skills, etc.
  • English Skills. In my opinion English skills of Russian outsourcing community are at the level you would expect them to be with a typical bell curve distribution and the median being at acceptable level.  Chances are you won’t have problems understanding developers and would be able to carry on a rich conversation with account managers and other client facing resources.
  • Rates. Rates of Russian development workforce vary greatly depending on location. Rates in T1 cities are very high, often making Russian outsourcing to be cost prohibitive. To deal with this issue many T1-city based vendors diversify by opening locations in small cities.  Rates for smaller city are as the standards of living in those cities – they fall off the cliff as soon as you move 100 miles outside of the tier one city boundaries.  However resulting rates continue to stay on a high side comparing to India’s.
  • Resource Turnover. Turnover tends to be on a low side comparing to India especially in a T2-T3 cities. The trend is however discouraging – according to what I hear from my network the turnover rate has bean steadily growing correlating to growing demand and increase in expected standard of living.

Let me close this post on a positive note covering one of the most important Pros of Russian outsourcing community – its Technical Capability. For many reasons Russia IT community in many cities in Russia offers above average technical capacity, innovation and creativity. That is particular notable for boutique vendors from St Petersburg, Moscow, Kiev, Minsk and Novosibirsk as well in the top echelone resources from lagre Russian outsourcers.

October 14, 2008 Posted by Nick Krym | Offshore Vendor Selection | , , , , | No Comments

Vendor Selection in China

This post is a summary of vendor selection trip to China made for a purpose of s/w outsourcing initiative for a midsized product company. The main focus of the trip was “profiling” of the vendors that made on a short list after a rather involved RFP process.

Profiling involved in-depth interviews of employees ranging from Sr. PM to Jr. QA analysts. I had a chance to interview over 60 people, and I believe that I had a chance to work with a somewhat fair sampling. I would expect that if employees were selected for interview completely randomly I would have the same professional skills ratings but English skills ratings would be substantially lower.

The table below presents a summary of my view on the software teams I’ve seen during the trip. Professional skills are rated from 0 to 10; 0 means no knowledge of the key subjects, 10 means exact or above expectations for the position. English skills are rated from 0 to 10, 0 means no knowledge, 10 means fluent (strong accent, minor grammar mistakes, etc. acceptable).

Position Professional Skills English Skills Comments
Account Management 5-8 6-9 I did not interview AMs per se, I had a plenty of time to observe their work though. Skills / understanding of AM practices were not at all impressive. While the hospitality was truly commendable understanding of AM activities was far from what I would expect from professional AM / sales / presales team. In particular the ability to listen and concentrate on my needs versus out of the box presentations and sales pitches was not demonstrated.
Project Management. 4-6 5-8 I interviewed 2 PMs, 2 were dreadful, one good, the rest were
semi-decent but junior. Most of the PMs had almost no theoretical
knowledge and border-line acceptable hands-on skills; only one was PMP
certified, unfortunately he needed an interpreter to communicate. Real
hands-on PM experience was ranging from 2 to 6 years. Most of the PM in
US terms could probably be ranked somewhere between a Project
Coordinator and Junior PM.
Business Analysis. 3-5 3-7 Unacceptable. I interviewed at least 8 of them and the only one I
would possibly consider was a junior Indian girl. Most of them had
moderate English skills, but still far less than you would expect from a
BA. Their skills in written English were notably better but they really
straggled in spoken language; understanding them was a challenge as
well. Their domain expertise was not impressive even for the projects
they worked on. Functional skills such as ability to gather requirements
were very poor. Technical skills such as data modeling skills were
practically non-existent.
Junior Developers (“coders”). 3-8 4-8 Developers range from very bad to pretty good. Most of them offered
very poor theoretical skills and narrow and shallow practical. Need to
be hand-picked, but there is a large pool to draw from. I would expect a
hit ratio of 1 out of 4. I saw great deal of desire to succeed and
multitude of signs of superb work ethics.
Senior Developers (“architects”) 3-6 3-5 Very poor, most of them at best would qualify for mid-level
developers. English skills are notably worse than juniors. The more
senior the person is the more difficult s/he is to understand. The only
good guy I met (would rate highly in Silicon Valley) required an
interpreter.
Technical

Leads

5-6 4-7 Mediocre. Probably not self sufficient on tasks requiring dealing
with complex technical issues. They seem to be generally a combination
of a mid-level developer with a junior PM. I would say on both PM and
technical accounts they are a notch lower than I would expect in the
USA. On the other hand I saw a very strong drive / desire to succeed
which could possibly compensate to some degree for the lack of
knowledge.
Junior QA, Black Box 6-9 5-10 Testing skills ranging from good to very good. English at pretty
decent level (most of them came from English studies or had lived in
English speaking countries). Most of the QA analysts I interviewed
seemed to have a great personality to position match.
Junior QA, Automation 4-5 5-7 Very small pool, most of them were mediocre at best with very
limited exposure to tools. Typical “record and play back” skill set.
Most of them had a career path of black box tester to an automation
engineer (no development background).
Senior
QA, Automation
4-5 3-5 Bad. I saw only four of them though; both skills and language were
below mediocre.
QA Lead 7-8 5-8 I saw 9 QA leads and all were OK, nothing spectacular but very
focused, detailed oriented, well organized, etc. Good grasp on QA
process (very specific to the company’s process though). Understanding /
grasp of QA automation at very basic level.
General Management 5-10 6-10 Very strong business leaders with outstanding work ethics and
commendable drive. Mostly ex-pats / returnees from Western countries /
Hong Kong / Singapore. However some of them were not professionally
strong as they seem to be able to get the jobs on the raising wave of
outsourcing mainly due to their western credentials. For example one of
the execs I met was a Ph.D. in theoretical physics with no prior
consulting / sales / software experience, very smart guy with very
little experience / exposure / understanding…

September 9, 2008 Posted by Nick Krym | Making Offshore Decision, Offshore Vendor Selection | , , , | 1 Comment

S/W Development Outsourcing: China vs. India

A few months ago I went through a vendor selection process for a technology company in SF Bay Area. The goal was to find a vendor that would become a long-term partner / a part of a local development team. My clients were set on considering only two countries – India and China.

The size of a potential outsourcing deal was fairly small: ~15 people. That would roughly correspond to $1M on annual basis. The size of engagement was still big enough to give us a chance to pick a company from a large pull of vendors who seemed to be interested. Here are some observations based on our analysis:

Rates

In both countries we saw a plenty of companies prepared to compete on price and go very low just to get the deal. Larger / more mature companies had notably higher rates, many of them with very similar message “We are by far not the cheapest but we are the best”. Considering companies we liked the rates for India depending on position and company were roughly between $20 and $35 an hour with blended rate for our team ~$30; the rates for China were roughly between $15 and $30 an hour with blended rate for our team ~$20;

Access to Resources

Access to resources in high-tech centers of India is getting increasingly complex, finding It talent ion cities such as Bangalore is almost as complex as in San Francisco. Outsourcing companies also have to compete with subsidiaries and offshore divisions of multinational corporations such as IBM, Microsoft, Accenture, etc.

We still were surprised with how difficult and slow the sourcing process was. It seemed that finding even mainstream roles such as .NET developers or Winrunner QA guys was practically impossible.

We found that access to resources in China was not as complex. It appeared that companies in China were able to staff up for a project 3-5 times faster than India companies. In example confirmed by the references it took 2 months to build a 30 FTE team versus 9 months for the same by Indian Tier 1 InfoSys.

Resource Quality

The companies with $5M-$20M revenue range (our target based on scope of outsourcing) fall in a group of 3rd to 5th tier companies with inevitable impact on their access to resources which is exacerbated by general scarcity of IT talent. So it was not a surprise for us to see very poor quality of resources. Our average “hit rate” (number of people we would consider for “hire” vs. people presented to us by the vendors) was 1 out of 4.

Companies with the same $5M-$20M revenue range are the first/second tier companies in China with top pick in off campus hiring as well as other methods of employee sourcing. We saw that as a solid, and possibly the most important, advantage for Chinese firms. However our hit ratio was even lower than in India – 1 out of 5.5 due to serious communication / language handicap.

Concentrating only on those resources who we considered potential “hires” we saw a decent blend of theoretical and practical knowledge with some diversity in skills / background / experience in India. Our potential “hires” in China showed rather weak theoretical knowledge across the board. Their practical skills were solid yet extremely narrow; most of the “good” people we talked with were “pigeonholed”, and did not seem to mind.

Employee Turnover

Getting honest information about turnover, retention and attrition seemed practically impossible so we got the numbers from unsolicited references rather than from the vendors.

Most of the large s/w outsourcing companies in India have turnover rates exceeding 30%. Attrition is particular high in large centers such as Bangalore, Hyderabad, Mumbai, New Delhi. For small companies it is not unusual to see offshore staff turnover rates exceeding 50%.

Turnover rates in China outsourcing industry are under 25%. The companies that were selected claimed to have attrition rate about 15%. We saw low attrition rates as probably one of the greatest advantages of China over India.

Communications

There is no comparison in communication skills of consulting work force in India and China. Command of English language for majority of people we interviewed in India was far stronger than mine. In China the situation was opposite, more so the more skilled and senior the resource the lower his/her language skills.

Language in just one of many dimensions of communication. There are many skills important for bridging cultural differences and communication gaps. When it comes to dealing with USA companies Indian consultants have a huge lead on Chinese in many aspects, just to name a few:

  • Body language / facial expressions – much easier to understand and follow
  • Overall presentation skills
  • Understanding of professional lingo
  • Grasp on general rules of professional communications and office etiquette

With communications being one of the most important aspects of majority outsourcing initiatives India have a huge lead on China.

Mindset & Work Ethics

Talking with a large group of reference accounts (including unsolicited) gave us an interesting insight into mindset and work ethics of development teams in India and China. Here are a couple things that people had general agreement upon when it comes to resources from Indian vendors:

  • A mindset of typical outsourcing company is oriented towards revenue / profit and is focusing employees towards “billing hours” rather than customer satisfaction or success of the project.
  • High turnover rates and general acceptance of job hopping have devastating impact on resource’s attitude and work ethics.
  • Majority of consultants have over-inflated expectations in terms of their seniority, type of work they should be doing, and a speed of promotion.

We heard much more favorable assessment of mind set and work ethics for Chinese workforce:

  • With aggressive market share oriented drive of China s/w outsourcers employees of these companies are focused on high productivity and customer satisfaction.
  • Workdays of 10-12 hours are not at all unusual (note that billing is typically negotiated on monthly basis with 8 hour workdays).
  • There is a strong prevalence of team values over individual.

The workforce work ethics present one of great advantages of the outsourcing companies in China. Combined with lower levels of flexibility in job market (partially due to the country’s political and economical structure) the work ethic to some degree offsets luck of knowledge and experience.

Total Cost of Outsourcing

And finally – Total Cost of Outsourcing (TCO). TCO is the cost that accommodates for the communication overhead, lower productivity, and all other costs that are not reflected in the rate. In a large degree assessing the TCO requires substantial experimental data with a specific vendor. Based on a survey of the references during this engagement, my own experience and experience of similar companies in terms of the structure and the scope of outsourcing we came up with interesting numbers presented below. The numbers are presented as a percentage of savings / losses over typical full time employee rates:

Project Type India China
Small R&D projects Loss 25 – 45% Loss 30 – 70%
Large R&D projects Loss 10 – 15% Loss 10 – 15%
Small mainstream projects Loss 10% – Saving 10% Loss 15% – Saving 15%
Large mainstream projects Savings 10 – 30% Savings 15 – 45%
Large QA (black box) projects Savings 15 – 30% Savings 25 – 50%

September 5, 2008 Posted by Nick Krym | Offshore Vendor Selection | , , , | 1 Comment

Pros and Cons of doing business in China

From 30,000’ view Pros and Cons of giving your outsourcing business to China could be summarized as

Pros

  • Comparatively low rates
  • Low attrition rates
  • Large pool of talent in many areas
  • Superb work ethics of the workforce
  • Well organized / highly disciplined organizations
  • Staff’s desire to succeed (learning and becoming stronger professional rather than pure career move)
  • Flexibility of the contract arrangements

Cons

  • Poor English skills
  • Weak grasp on western communications style, wide cultural gap
  • Poor theoretical knowledge in many key areas
  • Weak technical skills in comparison to the mainstream Silicon Valley resources
  • Limited access to resources in several key areas (e.g. business analysis, architecture)

For most outsourcing initiatives that I managed and consulted on Cons outweighed the Pros. Yet I hope that sooner or later I find the right project and a team in China to match, more so I believe that the balance of Pros and Cons is changing as we speak; in particular

  • Poor English skills – Chinese government and outsourcing companies are making very significant investments in English training.
  • Weak grasp on western communications style, wide cultural gap – A large number of expatriates returning to China with their families after living in the USA and other countries is reshaping culture of Chinese outsourcing.
  • Poor theoretical knowledge in many key areas – I am not seeing notable changes there; possibly due to the fact that majority of service buyers are not concerned with that issue.
  • Weak technical skills in comparison to the mainstream Silicon Valley resources – Using an old joke as a metaphor – Chinese vendors do not need to outrun Silicon Valley they just need to outrun Indian vendors.
  • Limited access to resources in several key areas (e.g. business analysis, architecture) – Influx of expatriates and attention to the issue should eventually take care of it.
Of course as Cons are being addressed China may lose some of its competitive advantage in Pros category – attrition rates, cost, etc. That remains to be seen.

I always thought that China was destined to win, and if I ever had any doubts they were eliminated after my trip to Shenzhen. When you see what a combination of strong hand of the government and grass root entrepreneurship can do to transform a small fishing village into a megapolis in just 35 years you start to believe that there is nothing that China can not achieve.

September 2, 2008 Posted by Nick Krym | Making Offshore Decision | , , , | No Comments

Protecting Data and IP when Outsourcing Offshore

Securing data when working with offshore is a well known yet a very challenging task. It’s especially serious if your company deals with financial or private data, such as ePHI (electronic protected health information). In some way though dealing with data protection in offshore scenario while complex is a straight forward task, especially for companies that are used to that kind of challenges in-house. Protecting Intellectual Property (IP) takes the challenge to a completely new level.

Risk of losing IP through offshore outsourcing is serious and real. I would venture to say that overall price tag related to IP loss in offshore outsourcing is measuring in billions. For example a friend of mine found himself out of the job after an IP ordeal with an outsourcing company in Eastern Europe. He was responsible for a product line in developer’s tools space in late nineties. He found a hired a group of vary talented engineers from Byelorussia. The requirements were coming from USA and development work was done 100% offshore. Source control and document repository were maintained offshore what seemed to be the right approach considering aggressive nature of the project and weak communication infrastructure. Cutting to the chase – when it came to transferring of the finished product into the hands of the owner the team in Byelorussia simply refused to do so. Initially the asked for some ridiculous amount of money but later on dropped out of negotiations, re-branded the product and took it to the market themselves…

To mitigate the risk you first need to understand channels of IP loss, here are the main few to consider:

  • There is clear possibility of malicious / criminal acts relevant to your IP. Your product idea could be stolen, repackaged and sold by the very partner you have entrusted. Not just idea, the source code, processes, documentation.
  • Even more probable scenario arises when a disgruntle or “entrepreneurial” employee of your vendor takes advantage of gaining access to your IP, source code, etc. Of course that could happen with your own staff; offshore just exacerbates the issue / increases the probability.
  • Immaturity of vendor infrastructure (physical security, network security, etc.) could become a reason for massive IP loss / data exposure. Insufficient physical, network security and data security opens up data and IP for hackers of all sorts.
  • Poor understanding of data and IP security, insufficient or non-existing security policy framework has the similar effect, often with even more severe consequences.
  • Casual treatment of IP security by your vendor. I remember visiting one offshore outsourcer in Eastern Europe. During a tour of facilities my guide brought me to office which had a number of expensive physical guards in place. We still went inside and my guide started – “here where we have super secret project with the company I can not name, they are a major search engine that rhymes with “frugal”, wink, wink. Those guys use our Ph.D’s to…”

What can you do to minimize / mitigate risks of IP loss with your outsourcing partner? That would be a next post

August 29, 2008 Posted by Nick Krym | Managing Offshore Engagements, Offshore Vendor Selection | , | 1 Comment

Top outsourcing risks

Putting development of your product or any other aspect of technology in the hands of a third party is certainly a risky proposition. To properly mitigate the risks of outsourcing one needs to understand the outsourcing landscape quite well. The top offshore outsourcing risks fall in several main categories. There is much to be said about each of the categories; I am planning to add more substance / clarifications /examples to each of the bullets below, as well as some ideas on risk mitigation. For now, here is the high level list:

Geopolitical

  • Government regulations, on the both sides of the equation
  • Political stability
  • Legal maturity

Security

Internal – Organization

Internal – Team and Personal

  • Loss of team support / respect / relationships with the team
  • Loss of team spirit / internal unease
  • Loss of key personnel / technology and business knowledge loss
  • Decrease in team’s productivity / commitment
  • Career impact
  • Lifestyle impact

Vendor capabilities

  • Financial Stability
  • Organizational maturity
  • Organizational commitment
  • Infrastructure (macro / micro view)
  • Technical capabilities
  • Ability to deliver
  • Personnel turnover

Joint responsibilities

  • Process confluence
  • Scope management
  • Geographical dispersion
  • Cultural differences
  • Knowledge transfer
  • Communications

August 26, 2008 Posted by Nick Krym | Making Offshore Decision | , , | 1 Comment

Not yet ready for China

I love travel and visit new places, even if that’s on a business trip. Needless to say that living “5 to 7” lifestyle (by 5 am Monday you are on a plane to the client and ~7 pm on Friday are back on your way home) grows old quickly, but my engineering leadership role delivers travel in just the right amount. So I was quite excited when I got a chance to go to China to meet with a few promising offshore outsourcing companies. Impressed by meetings with vendors’ execs I was looking forward to starting with one of their teams in China.

Jumping ahead, I have to tell you that out of my top three impressions of China one was far above the expectations, one was at par, and one was way below. The first one was food – sorry Chef Chu, Dragon Well, and Yank Sing – there is no better place for Chinese food than China. The impression that was exactly at par with my expectations was my prospect resources’ command of English – it was dreadful. That was not a big deal for me though – English is a second language and work in progress for me as well, plus quoting Clarence Darrow, “Even if you do learn to speak correct English, whom are you going to speak it to?”. The thing that fall way below my expectations and to large degree ruined the trip was technical skills of engineers I saw.

Having spent the last 10 years of my life in Silicon Valley I’ve been conditioned to be surrounded by people much smarter than I. A great majority of them came from Asia. So I was expecting to see people like Bhaskar H, John B, Mark Dao, Shao Fang, Harshal Deo, Michelle Sue, Ashish Mangla… I was expecting maybe not the same level of intelligence, grasp on technology, and knowledge of foundations but at least somewhat close. Instead, my vendors paraded in front of me dozens of engineers who could not explain what a polymorphism is, project managers who did not know how to use MS Project, business analysts with nothing but a desire to be one. A few days into my interview process going through at rate of ~20 30-min interviews per day I met only one decent PM who did not speak English at all, one good yet quite junior business analyst (she was an IIT grad and just moved to China with her husband), and a handful of barely acceptable engineers. My interviewing stats were:

Project Managers – 1 out of 9
Business Analysts – 1 out of 11
Senior Developers – 0 out of 6
Mid Level Developers 2 out 14
Junior Developers – 4 out of 9
QA Leads – 1 out of 4
Blackbox Testers – 4 out of 6
Automation Testers – 0 out of 7

After visiting four companies my mind was set and I switched to enjoying tours to Forbidden City, Bird’s Nest and Summer Palace; I climbed the Great Wall, took many pictures, and bought a bunch of souvenirs. I knew that would be a while before I see China again.

Having spent some time thinking through the reasons behind my failure to find the vendor I could probably attest to those commonly known:

  • China is relatively new to the IT outsourcing, in particular for US projects. There is a great deal of skills, experience and understanding that has not yet been built up.
  • Language is a natural and serious obstacle which China outsourcing companies need to invest a great deal.
  • Chinese outsourcers need to learn how to deal with a large variety of cultural differences to successfully compete (and not only on cost). I believe that they need to find their own style. While a lot could be taken from success of Indian vendors, just “cut and paste” would not work.

On my trip to China I also discovered a few things that I had not heard of before:

  • In Chinese education system getting an English major ranks bottom low vs. engineering or CS degrees. Inevitably it attracts the least talented students. However, in a race to address language handicap outsourcing companies recruit English major students for key development positions – project managers, business analysts, etc. No wonder none ob the BA I interviewed heard of UML…
  • While checking out offices of many outsourcing companies I noticed one thing in common: developers’ desks were perfectly clean – not a single book anywhere. I guess one of the reasons is in lack of relevant literature in native tongue. Reading O’Reilly in English is an uphill battle for many of engineers.
  • Most of the engineers I talked with gained all their knowledge on the projects they worked on, which is a great way to learn when it is one of the methods, not the only. Result is extremely narrow scope of knowledge / expertise.

So I guess I am not ready to send my work to China yet, while I really do want to. Why? That would be a great topic for another post.

August 23, 2008 Posted by Nick Krym | Making Offshore Decision | , , | No Comments

Offshore productivity: developers

Why is offshore developer’s productivity so much lower than one from your local employees? You would expect that for positions that require high communication overhead or for those tightly linked to difficult to acquire domain expertise, but why developer? Java, C#, C/C++ know no borders and far more international than Esperanto ever aspired to be. So why do we habitually see offshore developers offering productivity at fraction of what we expect in-house. I think there are many reasons for that, some transient some foundational. Here are just some of them, specifically applicable to India outsourcing:

  • Bangalore is a beautiful vibrant city; in some strange way it reminds me of San Francisco. No wonder it has become the heart and a synonym for IT outsourcing, one would be hard pressed to find a better place for India’s Silicon Valley. One of the parallels between Bangalore and SF Bay Area is abundance of technical jobs. Everyone and their brother are there from giants like Microsoft, IBM, and Siemens to small progressive engineering companies. Just look at a very small subset at http://en.wikipedia.org/wiki/List_of_IT_companies_in_Bangalore. So if you are a decent developer your job choices are practically unlimited. And let me ask you that – if due to your exceptional talents you had a choice of working for any software company in the world would you pick someone like Google or Infosys Technologies? It’s tough to get into someone’s mind yet I am sure that a choice between Google and Quadwave Consulting Pvt. Ltd. is statistically obvious. The corollary here is obvious as well: when it comes to pick of the litter even very successful outsourcers are in a long line behind a very large group of “the best companies to work for”.
  • One of the interesting phenomena is how cultural environment affects building the skills. What I have observed in several geo areas is that the culture is “against” the learning progress; that is especially true for consulting companies. For example, many consulting companies have a business development track vs. expert track, the first one being compensated substantially better. In that case a bright person is pushed either to take a biz dev track or bail out of the company when his / her earnings come hits the ceiling. Broader cultural issues have even more profound effect, for example I noticed that in India one’s personal success is measured in number of people working “under you”, most of the time that means that one’s skills would be used mostly on management tasks and not tuned to acquiring new skills / knowledge / etc. Very much a cliché – promoting a good engineer to a manger means losing a good engineer and gaining a bad manager. The impact of that trend is quite devastating for buyer especially with smaller sized teams – how can you get a solid tech lead for your team if it is only 3-4 people while someone with 5 years experience could be managing a team of 30? That trend is exacerbated by scarcity of high-level resources every consulting company faces: if a practice gets their hands on great resource, he or she must be leveraged across large group of mediocre resources.
  • I realized another angle to the topic when building my own consulting / outsourcing teams. Managing engineers is not an easy task, managing bright engineers is extremely complex. So if your main goal is generating a revenue stream (a.k.a. “putting buns in seats”) you may take an “expert” route and deal with celebrity personalities and never-ending need for motivation or build your team with “a middle of the road” resource in mind and live happily ever after. The later seems like a much easier route and that’s why many offshore resource augmentation companies are in the business of “selling mediocrity in bulk”. What is the difference in productivity between star and mediocre software developer? Some say that it’s order of several magnitudes, and I can attest to it.

August 22, 2008 Posted by Nick Krym | Making Offshore Decision, Managing Offshore Engagements | , | No Comments

Outsourcing Myths: cost advantage

On a surface that seems obvious: even with dollar fall, rising cost of living in India, China and especially eastern Europe the hourly rates continue to be much lower than those you have to pay in the states. For example, a mid level java developer would roughly cost you:

SF Bay Area, full time $60/hr
SF Bay Area, contractor $80/hr
Bangalore, India $25/hr
St. Petersburg, Russia $30/hr
Shenzhen, China $20/hr

So for every contractor in your San Mateo office you can put 4 in Shenzhen or 2 in St. Petersburg. Or, looking from savings angle, instead of paying a team of 8 engineers roughly $80K a month you can pay less than $30K in China and get yourself a hefty raise.

However, let’s take a look at real cost of outsourcing. The first rate killer is productivity: in my experience on average productivity of resources in India would be 50% of what you would get with local mid level resources, for senior developers it would be at best 75%, and for juniors you could potentially see it as good as 80%. To achieve that level of productivity you would need to put above average efforts in shaping your team and have considerable amount of luck. Typically getting a good senior developer is only possible if s/he is managing is at least 20-25 resources. Also be prepared that understanding of seniority may greatly vary from what is considered a norm locally, and in that light delivering another productivity hit. Let’s imagine that you are planning to outsource a 5 member team that includes one tech lead, two mid-level java developer and relatively junior ones. Your cost for the team would be roughly:

Role Level Experience QTY Salary Aprox Rate Monthly Cost
Tech Lead Senior 10+ 1 $140,000.00 $ 82.60 $ 14,537.60
Developer Mid 7+ 2 $100,000.00 $ 59.00 $ 20,768.00
Developer Junior 3+ 2 $ 75,000.00 $ 44.25 $ 15,576.00

Total:

$ 50,881.60

What would it cost you in Noida? Let’s assume that you got lucky in finding a tech lead for the team and the rest of the members were typical developers you could find out there.

Role Level Experience QTY Salary Aprox Rate Monthly Cost
Tech Lead Senior 10+ 1 N/A $ 32.00 $ 5,632.00
Developer Mid 7+ 2 N/A $ 28.00 $ 19,712.00
Developer Junior 3+ 2 N/A $ 25.00 $ 17,600.00
PM Mid 5+ 1 N/A $ 30.00 $ 5,280.00

Total:

$ 48,224.00

Note that team in India also includes full time PM which is not surprising for a team of 9. The result is a staggering 5% of cost savings. Your actual numbers could be different based on your negotiating skills, but the dynamics won’t change.

I am sure that many outsourcing companies would dispute my assessment of productivity. Well, let me give you a couple examples:

  • We stopped a development project (CICS adapter) being developed by team of five “very senior engineers” from Mastech after ~3 months due to unacceptable quality of deliverables. 2 months later the adapter was developed by a single developer we had on staff.
  • We had to stop and take in-house a handheld development project (.NET) being delivered by a senior developer and several mid-level ones from MindTree due to low quality of code and extremely low productivity. This project was later delivered by a single mid-level developer with oversight from a senior developer in time shorter than it had taken us to ramp up the team in Bangalore.

The list of examples can go on and on, they are just examples, however, considering that I could not offer any examples of an opposite nature, there is possibly a trend here. The question is why offshore developer’s productivity is so low? I’ll put my thoughts on in a separate post.

August 22, 2008 Posted by Nick Krym | Making Offshore Decision | , | 2 Comments