Idiot Savant
It’s been almost two years since the story below shook up my organization yet it’s still quite fresh in my memory. This story stands out as rude reminder of how complex IP protection is and how many traps you need to consider. Even though the story is a bit old I still changed the names of the participants to protect the innocent…
Ravindra Gupta a senior java architect from a well respected local high-end consulting firm ThinkBig impressed us the first time we met him. He seemed to be exactly the person we’d been looking for a long time. With little hesitation we assigned him a challenging task in a user management and administration (UMA) space. He flew in our SF location and spent a couple weeks going through the analysis of the issue, discussions with our staff and me.
I consider UMA tasks some of the most challenging and was happy to have a very impressive guy deal with them. We spent days educating Ravi on functional aspects of the UMA, its issues, technical challenges, and numerous tricks and traps. I was very impressed with Ravi grasp and progress with it. After a few weeks in the office Ravi took off to work from home at the East Cost.
The year end frenzy took my mind of UMA and Ravi’s work. Ravi’s brief status reports and scant communications did not bother me much especially considering that he was apparently making good progress and we were expecting detailed technical review session right after the holidays. Unfortunately though the meeting never not took place, instead one of the first emails of ’07 was one from Ravi:
From: Ravindra S. Gupta [mailto:rgupta@thinkbigconsulting.com]
Sent: Saturday, January 06, 2007 1:33 PM
To: Nick Krym
Subject: U.M. Design Copyright
Hello Nick,
In the chaos of the holidays, I was fortunate to find some time to myself. This yielded an insight: I have to stop selling my copyright for the price of a copy. “R.S.G. Consulting” was therefore founded (finally) on 1/1/2007.
Consequently, the existing contract (made with ThinkBig) is no longer acceptable, and I want to proceed forward under the following general agreement:
I own the copyright to all parts and whole of the generic Access Control System (RACS) that I am creating. Medem does not pay for the development of RACS, but may buy a very favorably priced license. However, it is under no obligation to do so.
To demonstrate suitability-of-use, I will configure the RACS to support up to five (5) use-cases chosen by Medem, at no cost to Medem.
Should Medem then buy a license (still no obligation), it may buy my professional services to fully or partially configure the RACS for use with the new Medem system. All work done towards this task will belong to Medem (i.e. will bear Medem’s copyright), with exception of any modifications made to the RACS (which will bear my copyright). However, Medem may instead choose to have its own developers or other consultants configure the RACS.
The attached document describes the Pricing model, Availability, etc.
The design that’s taking shape is beautiful in its simplicity and power of expression. It will more than accommodate Medem’s needs for years to come.
Please feel free to contact me at any time.
Sincerely,
Ravi
R.S.G. Consulting
I red the email and picked up phone calling Mike, CEO of ThinkBig.
A month later, after the dust settled, legal bills were paid and separation agreement signed Mike and I were seeping coffee at our favorite spot at Montgomery and Bush. Mike was going in rounds about the fact that someone he’d been working for over ten years would do anything like that. In my mind there was just one answer, an expression that I had recently learned: “Idiot Savant”
Pros and Cons of doing business in China
From 30,000’ view Pros and Cons of giving your outsourcing business to China could be summarized as
Pros
- Comparatively low rates
- Low attrition rates
- Large pool of talent in many areas
- Superb work ethics of the workforce
- Well organized / highly disciplined organizations
- Staff’s desire to succeed (learning and becoming stronger professional rather than pure career move)
- Flexibility of the contract arrangements
Cons
- Poor English skills
- Weak grasp on western communications style, wide cultural gap
- Poor theoretical knowledge in many key areas
- Weak technical skills in comparison to the mainstream Silicon Valley resources
- Limited access to resources in several key areas (e.g. business analysis, architecture)
For most outsourcing initiatives that I managed and consulted on Cons outweighed the Pros. Yet I hope that sooner or later I find the right project and a team in China to match, more so I believe that the balance of Pros and Cons is changing as we speak; in particular
- Poor English skills – Chinese government and outsourcing companies are making very significant investments in English training.
- Weak grasp on western communications style, wide cultural gap – A large number of expatriates returning to China with their families after living in the USA and other countries is reshaping culture of Chinese outsourcing.
- Poor theoretical knowledge in many key areas – I am not seeing notable changes there; possibly due to the fact that majority of service buyers are not concerned with that issue.
- Weak technical skills in comparison to the mainstream Silicon Valley resources – Using an old joke as a metaphor – Chinese vendors do not need to outrun Silicon Valley they just need to outrun Indian vendors.
- Limited access to resources in several key areas (e.g. business analysis, architecture) – Influx of expatriates and attention to the issue should eventually take care of it.
| Of course as Cons are being addressed China may lose some of its competitive advantage in Pros category – attrition rates, cost, etc. That remains to be seen.
I always thought that China was destined to win, and if I ever had any doubts they were eliminated after my trip to Shenzhen. When you see what a combination of strong hand of the government and grass root entrepreneurship can do to transform a small fishing village into a megapolis in just 35 years you start to believe that there is nothing that China can not achieve. |
 |
Offshore Risk: Cost-reduction expectations
Establishing high cost-reduction expectations is one of the most serious traps a technology leader can get him/herself into. If the only reason you are going offshore is cost savings – my best advice would be – stop right there! If you are very good at utilizing offshore you may realize 30% savings on somewhat sizable initiatives, and you still will need a lot of luck. That’s aside even if you do understand the paradigm of cost savings you still have to establish appropriate expectations with your execs / peers / team. Failure to establish correct expectations results in insufficient budgets, often in a collapse of the entire outsourcing initiative with a serious ripple effect.
I receive emails from one of Beyondsoft (China) sales execs on a pretty much monthly basis. While his tenacity is commendable his message is totally ludicrous, here is one of his emails:
Dear Nick,
I know your schedule is very tight, but I really hope we have an opportunity to share our ideas on how to help you decrease cost by 300% in next 12 months.
I thought that’s a good opportunity coz you are in Beijing now, and our meeting would make you more impressive.
Looking forward to your early reply.
Best regards,
George Tong
300% wow! where do I sign!? Think about your execs who are continuously spammed with such messages. Direct mail, articles, whitepapers, case studies and so on conveniently delivered to your boss’s ear scream about potential saving from offshore. Setting appropriate expectation on your part will be a balancing act of delivering a bad news without sounding like a sandbagger. Here is a presentation approach I found somewhat successful in setting my audience’s expectations at a reasonable level.
Start with debunking the Myth of Cost Savings
- What vendors are telling us
- Couple genuine offshore horror stories
- Rates vs. True Cost of Outsourcing
Change audience focus to specific challenges / reasons for outsourcing, e.g.
- Time to market
- Access to specific resource type
- Refocusing internal resources
Setup SMART (specific measurable actionable result-oriented and time-bound) goals for outsourcing, e.g.
- Move 100% maintenance of product X to Worksoft team by 5/15/210
- Deliver 50 functional points by ZenSar’s team by 9/20/2009
Another pointer – my recommendation is to setup an expectation that there are NO cost savings and work in terms of alternative delivery benefits rather than cost. For example:
- The project Odessa requires 5 FTE for 10 months.
- While we do have a budget for it we do not have the resources.
- Finding skilled developers and QA engineers is likely to take us over 3 months and we will need to train them for about 2 months.
- To save the ramp up time we are going to us MindTree team.
- The budget allows us enough resources to deliver the project under 12 months.
There is another trap here – what if the team you recommended can not deliver on time? Well, that’s not at all inconceivable, yet easier to control and deal with.
Top outsourcing risks
Putting development of your product or any other aspect of technology in the hands of a third party is certainly a risky proposition. To properly mitigate the risks of outsourcing one needs to understand the outsourcing landscape quite well. The top offshore outsourcing risks fall in several main categories. There is much to be said about each of the categories; I am planning to add more substance / clarifications /examples to each of the bullets below, as well as some ideas on risk mitigation. For now, here is the high level list:
Geopolitical
- Government regulations, on the both sides of the equation
- Political stability
- Legal maturity
Security
Internal – Organization
- Cost-reduction expectations
- Organizational commitment
- Organizational maturity – multiple aspect of being ready to outsource
- Loss of team support / respect / relationships with the team
- Loss of team spirit / internal unease
- Loss of key personnel / technology and business knowledge loss
- Decrease in team’s productivity / commitment
- Career impact
- Lifestyle impact
Vendor capabilities
- Financial Stability
- Organizational maturity
- Organizational commitment
- Infrastructure (macro / micro view)
- Technical capabilities
- Ability to deliver
- Personnel turnover
Joint responsibilities
- Process confluence
- Scope management
- Geographical dispersion
- Cultural differences
- Knowledge transfer
- Communications
My reasons to outsource
I mentioned already the top reasons for offshore outsourcing typical for many organizations; let me list some of my own:
- Diversity. Diversity in terms of bringing individual contributor with different background into the team often means a tremendous increase in productivity. A healthy portion of resources with different education, practical background, and way of operating could bring a fresh breath of air in stagnating organization. Also “diversifying” your portfolio of resources might help a great deal to deal with micro factors affecting employment / recruitment landscape of a specific geography.
- Education. In countries such as India, Russia, China you find many people who value education to much higher degree than we do in the states. On one of my teams from St. Petersburg a majority of developers had at least MS and over 40% had Ph.D. That including QA engineers! Needless to say the brain power of the team was absolutely amazing.
- Work Ethics. That doesn’t go across all geographies and companies, but fortunately you still can find outsourcing organizations with resources who’s work ethics are far superior to what you find for example in corporate America.
- Talent Pool. Some outsourcing organizations instead of typical “selling mediocrity in bulk” build their team with top notch experts and people with exceptionally high IQ. Building such a team, no matter in which area of the world takes very long time.
- Processes. Getting process right is time consuming and costly. When ISO or CMM processes are a requirement it’s often much easier to build relationship with a subcontractor who already has those in place.
- Project Management. Project and program management is often something that a small software organization can not afford (or more often VPE can’t sell his execs / team on the need for it). Many, especially Indian vendors have that in perfect shape.
- Cost. While I do not believe that offshore guarantees cost savings I do believe that there is a huge potential there especially with careful execution of multi-sourcing or/and micro-sourcing strategies.
Not yet ready for China
I love travel and visit new places, even if that’s on a business trip. Needless to say that living “5 to 7” lifestyle (by 5 am Monday you are on a plane to the client and ~7 pm on Friday are back on your way home) grows old quickly, but my engineering leadership role delivers travel in just the right amount. So I was quite excited when I got a chance to go to China to meet with a few promising offshore outsourcing companies. Impressed by meetings with vendors’ execs I was looking forward to starting with one of their teams in China.
Jumping ahead, I have to tell you that out of my top three impressions of China one was far above the expectations, one was at par, and one was way below. The first one was food – sorry Chef Chu, Dragon Well, and Yank Sing – there is no better place for Chinese food than China. The impression that was exactly at par with my expectations was my prospect resources’ command of English – it was dreadful. That was not a big deal for me though – English is a second language and work in progress for me as well, plus quoting Clarence Darrow, “Even if you do learn to speak correct English, whom are you going to speak it to?”. The thing that fall way below my expectations and to large degree ruined the trip was technical skills of engineers I saw.
Having spent the last 10 years of my life in Silicon Valley I’ve been conditioned to be surrounded by people much smarter than I. A great majority of them came from Asia. So I was expecting to see people like Bhaskar H, John B, Mark Dao, Shao Fang, Harshal Deo, Michelle Sue, Ashish Mangla… I was expecting maybe not the same level of intelligence, grasp on technology, and knowledge of foundations but at least somewhat close. Instead, my vendors paraded in front of me dozens of engineers who could not explain what a polymorphism is, project managers who did not know how to use MS Project, business analysts with nothing but a desire to be one. A few days into my interview process going through at rate of ~20 30-min interviews per day I met only one decent PM who did not speak English at all, one good yet quite junior business analyst (she was an IIT grad and just moved to China with her husband), and a handful of barely acceptable engineers. My interviewing stats were:
Project Managers – 1 out of 9
Business Analysts – 1 out of 11
Senior Developers – 0 out of 6
Mid Level Developers 2 out 14
Junior Developers – 4 out of 9
QA Leads – 1 out of 4
Blackbox Testers – 4 out of 6
Automation Testers – 0 out of 7
After visiting four companies my mind was set and I switched to enjoying tours to Forbidden City, Bird’s Nest and Summer Palace; I climbed the Great Wall, took many pictures, and bought a bunch of souvenirs. I knew that would be a while before I see China again.
Having spent some time thinking through the reasons behind my failure to find the vendor I could probably attest to those commonly known:
- China is relatively new to the IT outsourcing, in particular for US projects. There is a great deal of skills, experience and understanding that has not yet been built up.
- Language is a natural and serious obstacle which China outsourcing companies need to invest a great deal.
- Chinese outsourcers need to learn how to deal with a large variety of cultural differences to successfully compete (and not only on cost). I believe that they need to find their own style. While a lot could be taken from success of Indian vendors, just “cut and paste” would not work.
On my trip to China I also discovered a few things that I had not heard of before:
- In Chinese education system getting an English major ranks bottom low vs. engineering or CS degrees. Inevitably it attracts the least talented students. However, in a race to address language handicap outsourcing companies recruit English major students for key development positions – project managers, business analysts, etc. No wonder none ob the BA I interviewed heard of UML…
- While checking out offices of many outsourcing companies I noticed one thing in common: developers’ desks were perfectly clean – not a single book anywhere. I guess one of the reasons is in lack of relevant literature in native tongue. Reading O’Reilly in English is an uphill battle for many of engineers.
- Most of the engineers I talked with gained all their knowledge on the projects they worked on, which is a great way to learn when it is one of the methods, not the only. Result is extremely narrow scope of knowledge / expertise.
So I guess I am not ready to send my work to China yet, while I really do want to. Why? That would be a great topic for another post.
Offshore productivity: developers
Why is offshore developer’s productivity so much lower than one from your local employees? You would expect that for positions that require high communication overhead or for those tightly linked to difficult to acquire domain expertise, but why developer? Java, C#, C/C++ know no borders and far more international than Esperanto ever aspired to be. So why do we habitually see offshore developers offering productivity at fraction of what we expect in-house. I think there are many reasons for that, some transient some foundational. Here are just some of them, specifically applicable to India outsourcing:
- Bangalore is a beautiful vibrant city; in some strange way it reminds me of San Francisco. No wonder it has become the heart and a synonym for IT outsourcing, one would be hard pressed to find a better place for India’s Silicon Valley. One of the parallels between Bangalore and SF Bay Area is abundance of technical jobs. Everyone and their brother are there from giants like Microsoft, IBM, and Siemens to small progressive engineering companies. Just look at a very small subset at http://en.wikipedia.org/wiki/List_of_IT_companies_in_Bangalore. So if you are a decent developer your job choices are practically unlimited. And let me ask you that – if due to your exceptional talents you had a choice of working for any software company in the world would you pick someone like Google or Infosys Technologies? It’s tough to get into someone’s mind yet I am sure that a choice between Google and Quadwave Consulting Pvt. Ltd. is statistically obvious. The corollary here is obvious as well: when it comes to pick of the litter even very successful outsourcers are in a long line behind a very large group of “the best companies to work for”.
- One of the interesting phenomena is how cultural environment affects building the skills. What I have observed in several geo areas is that the culture is “against” the learning progress; that is especially true for consulting companies. For example, many consulting companies have a business development track vs. expert track, the first one being compensated substantially better. In that case a bright person is pushed either to take a biz dev track or bail out of the company when his / her earnings come hits the ceiling. Broader cultural issues have even more profound effect, for example I noticed that in India one’s personal success is measured in number of people working “under you”, most of the time that means that one’s skills would be used mostly on management tasks and not tuned to acquiring new skills / knowledge / etc. Very much a cliché – promoting a good engineer to a manger means losing a good engineer and gaining a bad manager. The impact of that trend is quite devastating for buyer especially with smaller sized teams – how can you get a solid tech lead for your team if it is only 3-4 people while someone with 5 years experience could be managing a team of 30? That trend is exacerbated by scarcity of high-level resources every consulting company faces: if a practice gets their hands on great resource, he or she must be leveraged across large group of mediocre resources.
- I realized another angle to the topic when building my own consulting / outsourcing teams. Managing engineers is not an easy task, managing bright engineers is extremely complex. So if your main goal is generating a revenue stream (a.k.a. “putting buns in seats”) you may take an “expert” route and deal with celebrity personalities and never-ending need for motivation or build your team with “a middle of the road” resource in mind and live happily ever after. The later seems like a much easier route and that’s why many offshore resource augmentation companies are in the business of “selling mediocrity in bulk”. What is the difference in productivity between star and mediocre software developer? Some say that it’s order of several magnitudes, and I can attest to it.
Outsourcing Myths: cost advantage
On a surface that seems obvious: even with dollar fall, rising cost of living in India, China and especially eastern Europe the hourly rates continue to be much lower than those you have to pay in the states. For example, a mid level java developer would roughly cost you:
SF Bay Area, full time $60/hr
SF Bay Area, contractor $80/hr
Bangalore, India $25/hr
St. Petersburg, Russia $30/hr
Shenzhen, China $20/hr
So for every contractor in your San Mateo office you can put 4 in Shenzhen or 2 in St. Petersburg. Or, looking from savings angle, instead of paying a team of 8 engineers roughly $80K a month you can pay less than $30K in China and get yourself a hefty raise.
However, let’s take a look at real cost of outsourcing. The first rate killer is productivity: in my experience on average productivity of resources in India would be 50% of what you would get with local mid level resources, for senior developers it would be at best 75%, and for juniors you could potentially see it as good as 80%. To achieve that level of productivity you would need to put above average efforts in shaping your team and have considerable amount of luck. Typically getting a good senior developer is only possible if s/he is managing is at least 20-25 resources. Also be prepared that understanding of seniority may greatly vary from what is considered a norm locally, and in that light delivering another productivity hit. Let’s imagine that you are planning to outsource a 5 member team that includes one tech lead, two mid-level java developer and relatively junior ones. Your cost for the team would be roughly:
| Role | Level | Experience | QTY | Salary | Aprox Rate | Monthly Cost |
| Tech Lead | Senior | 10+ | 1 | $140,000.00 | $ 82.60 | $ 14,537.60 |
| Developer | Mid | 7+ | 2 | $100,000.00 | $ 59.00 | $ 20,768.00 |
| Developer | Junior | 3+ | 2 | $ 75,000.00 | $ 44.25 | $ 15,576.00 |
|
Total: |
$ 50,881.60 | |||||
What would it cost you in Noida? Let’s assume that you got lucky in finding a tech lead for the team and the rest of the members were typical developers you could find out there.
| Role | Level | Experience | QTY | Salary | Aprox Rate | Monthly Cost |
| Tech Lead | Senior | 10+ | 1 | N/A | $ 32.00 | $ 5,632.00 |
| Developer | Mid | 7+ | 2 | N/A | $ 28.00 | $ 19,712.00 |
| Developer | Junior | 3+ | 2 | N/A | $ 25.00 | $ 17,600.00 |
| PM | Mid | 5+ | 1 | N/A | $ 30.00 | $ 5,280.00 |
|
Total: |
$ 48,224.00 | |||||
Note that team in India also includes full time PM which is not surprising for a team of 9. The result is a staggering 5% of cost savings. Your actual numbers could be different based on your negotiating skills, but the dynamics won’t change.
I am sure that many outsourcing companies would dispute my assessment of productivity. Well, let me give you a couple examples:
- We stopped a development project (CICS adapter) being developed by team of five “very senior engineers” from Mastech after ~3 months due to unacceptable quality of deliverables. 2 months later the adapter was developed by a single developer we had on staff.
- We had to stop and take in-house a handheld development project (.NET) being delivered by a senior developer and several mid-level ones from MindTree due to low quality of code and extremely low productivity. This project was later delivered by a single mid-level developer with oversight from a senior developer in time shorter than it had taken us to ramp up the team in Bangalore.
The list of examples can go on and on, they are just examples, however, considering that I could not offer any examples of an opposite nature, there is possibly a trend here. The question is why offshore developer’s productivity is so low? I’ll put my thoughts on in a separate post.
Top reasons for outsourcing
Whenever I have to introduce using offshore to executives I start with a slide with has title “Good reasons for going offshore”. The slide is otherwise empty. With all the headache it will cause why would I go offshore if I do not absolutely have to? I have been using offshore vendors since ‘92 and there is no place on my body which doesn’t have scars left by that experience – from grotesque stabs on the back to burned fingers. And yet again and again I find myself using offshore vendors and recommending them to my clients and partners. So what are the main “bad” reasons for using offshore? What typically drives companies to consider offshore outsourcing? For many companies those reasons could be roughly grouped into 3 categories:
Reducing operations cost, some most obvious examples would be:
- Lower resource cost; this topic deserves a separate discussion but for now let me just say that if you are good at utilizing offshore you may realize 25-30% savings;
- G&A savings (benefits, office space, utilities, etc.);
- Other resources related savings (reasonable severance, resource add-on acquisition cost, training cost, etc.).
Reducing time to operations / time to market, some examples include:
- Access to existing pool of resources vs. hiring; I will need to touch upon that subject in more details, for now let me just say that that in some cases that is true;
- Establishing development / operations infrastructure vs. using existing or building upon existing;
- Access to specific skills / domain expertise / methodology, etc.
Solving specific organizational issues, a lot of diverse and unique items here, here are just a few common examples:
- Outsourcing legacy maintenance to motivate developers by moving them onto more glorious tasks. That is a well known double-edged sward though.
- Dealing with sawmill of resource demand. Need for resources goes up and down if you staff at the top level your resource utilization suffers, if you target the bottom you can not react to market needs. Proper offshore supplement can help alleviate the issue.
- Risk mitigation, for example: should you hire for dealing with substantial spike in demand? Is the change permanent or when the novelties ware off you will be facing RIWFs? Outsourcing could be a better way to handle the spike.
- In some way the same idea as above but put into terms very clear and dear to hearts of CFOs – eliminating fixed cost
- Using third party operational expertise, existing processes, certifications, etc.
While in a way each of the reasons above could be enough to consider offshore in my view you need more than one reason to actually go offshore. The risks and penalties are just too high. And that deserves a separate discussion.
eBook
The Pragmatic Outsourcer series consists of 5 volumes, each covering a different phase of the outsourcing process. Volume 1: Making the Decision is available today for your free download. Volumes 2 through 5 will be available during 2010.
Read more
Tools & Links
