For some reason this month turned out the highest traffic in the history of this blog. It is particular interesting considering that I have not been doing a good job keeping it updated. So I feel obligated to add at least one post before December.
Let me cover some progress I made with my offshore provider I inherited through the M&A process. In particular let me cover some metrics we have established in order to control the relationship. This set of metrics is a preliminary set based primarily on what my vendor could easily retrieve from the tracking systems they already have in place.
A couple months ago we agreed on several metrics and established high level benchmarks. I believe that you can not enforce benchmarks on day one. You should setup goals and see whether they are reasonable to achieve considering multiple factors ranging from abilities of vendor and your own organization to quality of the tools that are used to collect the info.
Here are the metrics we have collected so far:
For sustenance / maintenance tasks –
- Average turn around time for issues by priority. That metric helps us control compliance with SLA we have in place.
- End-to-end response time for issues by priority. That metrics also helps us control compliance with SLA.
- Resource utilization. Helps both parties understand resource utilization / allocation / team effectiveness.
- Defect removal efficiency. Somewhat generic QA metric helping understand the quality of the deliverables.
- Effort / Actuals variance. Metric designed to control estimating process and SLA compliance as well.
For development tasks the metrics are slightly different –
- On time delivery index. Obvious SLA relevant metric.
- Effort Variance. Same as above just substantially more important for development projects.
- Number of Code Review Comments. That one was easily available so we decided to track it. I do not have expectations related to its utilization though. The data behind the metric is just too ambiguous.
- Requirements clarity index. I am interested in how this one pans out. It could be a good metric for measuring the quality of requirements process.
The metrics delivered to on-site team by the vendor once a month at executive reviews. There will be more metrics that we will track, but that’s not bad for the beginning. I will keep you posted, in meanwhile here is how these metrics look in the monthly report (one of the slides for a specific project).