How to evaluate software development company performance
/Do you find that your software development company is taking too long to deliver a project? Do they lack creativity and innovation? Would you like a partner who can provide accountability and transparency in their work? If so, read on.
As a custom software development company that have worked with hundreds of clients all over the world we have come across all kinds of situations where multiple parties are involved in creating a software platform. The performance of each party is crucial for the overall pace and success of the software project. And we have advised our customers about how to monitor and assess performances of their vendors - including ourselves! In today’s post I will summarize some of the biggest tips we have for our customers to assess the performance of a software developer.
The basics
In order to evaluate the performance of a software development company, there are four criteria worth considering:
speed, price, quality & service
Here's how these factors stack up for one of our clients:
-Speed: Slow delivery times lead to lost revenue as well as frustrated customers. Our client has already seen an average 30% increase in productivity from the new vendor we partnered with them on this project.
-Price: You get what you pay for! The old adage applies here, however there is a big mitigating factor here - location. If you are using a vendor at a relatively lower cost location such as ourselves based in Bangladesh you have the opportunity to find a price that is lower without compromising on the quality.
-Quality: This is the the most important one. In the world of software the only thing that really matters is quality. And you could compromise on pretty much all the other factors but not on the quality.
-Service: How’s the requirement analysis, the communications during the build, the delivery, the promises kept and the promises broken? All of these go towards the service metric.
The performance measurement matrix
The performance measurement is not a linear operation. That is to say it is not a single number you are looking at. For example a company that is great at quality might be really poor at speed. Does that mean it’s a badly performing vendor? The only sane way of getting a full view of the performance is to create a matrix that goes over the issues at stake and measures each one of them against the four basic dimensions of performance mentioned above.
One thing to note before I jump into this: there are hundreds of “scientific” way of measuring performance of a software development vendor. This is a topic that is favourite with efficiency specialists, university computer engineering departments and large multinationals with huge investments in IT. If you really want to get to know some of the research work that is done in this space a great starting point is the highly academic yet useful book generalized towards engineering organization performance: Performance Measurement and Management for Engineers or if you prefer a google search will lead you to a pages and pages of approaches towards measurements. What you’ll soon learn is that there is no standardized foolproof way to do this. At the end you end up with a lot of hand wavy metrics. All the big names in software has said something or other about the futility of measuring performance in software projects! Here’s one I like a lot:
“So while I see the value of measuring individual performance in research settings, I think its difficult to find cases in which the effort is justified on real projects.”
OK, now that I’ve got that out of the way, let me go over what we suggest. Our approach is pragmatic. We accept that no measurement scheme will be perfect, so all we can do is layout some basic information in a way that’s easier to visualize and let project owner do their own assessment based on that. So that leads us to the matrix I was referring to.
The columns in the matrix are the four basics - speed, price, quality & service and on the rows are the questions or issues that matter to the project or to a particular company. In this super simple method we take away the pain of running elaborate surveys, questionnaires, software line of code analysis etc. that some approaches use and use a very intuitive way of quickly getting to some feelings about the overall performance. Note that the number of rows, that is the issues, actually vary project to project, company to company. And that makes perfect sense, what matters to one project may not be an issue at all to worry about in another.
Here’s a sample matrix that we did for one of our customers recently. They had a total of 28 issues that they tracked with this, beyond the ones shows in the screenshot the rest were specific the product features, stuff like “All screens should update with animation”, etc. These made sense for this particular customer as they thought this was essential for the assessment of performance, but for many other companies such a granular level or such a narrow scope for the assessment wouldn’t make sense. That is the beauty of a simple matrix approach such as this one - it is flexible and can be adapted to the need of a specific project/customer/vendor situation.
When it comes to evaluating software development company performance, there are a number of factors that can help you determine if the vendor is working in your best interest. I’ve outlined our suggested way that seems to work very well for software projects. You can use this at regular intervals (we suggest every 6 months) to check the performance and detect signs of change. The change in scores from previous ones give you the signs of a change in performance - for the better or for the worse. Do any of these signs apply to your current vendor? If need help give us a shout at info@kaz.com.bd to learn how our team might be able to help you.
