How to test a new outsourcing partner?

In the world of software development projects there are many ifs, buts and maybes. It's hard to find absolutes. It's next to impossible to give a formula for success. But I think at least the answer to the question "how should I test a new outsourcing partner?" is one of the rare cases where you can confidently say you know the absolute correct answer! It is equivocally:

Start with a small test project or a prototype building phase.

Here are some examples of this small project that we, at Kaz Software, have suggested, to our clients, to test us out!

• A test project - made up just like a test to try the new partners out. Ideally you should know what outcome you'd like so that you can judge (just like an exam) the result. Even consider doing this test internally with your team too, so that you can compare. 
• A tool that you needed built - this may be something thing you always needed but could never spare enough dev time with your team to do it. Say for example - a tool that helps HR generate some monthly reports. The tool should have some challenges that you want to check how the new team solves. Nice thing with a tool is that you probably will get a useful tool out of this testing period.
• A prototype - this is a really good approach for start-ups. Get the team to build a quick-n-dirty prototype of the actual application that you want them to build (at least a subset of it that is). This has the added benefit that you can judge how much they captured your specs and also lets you use the prototype for demos and future spec clarifications. And if you continue with that team the domain knowledge already picked up is really a big thing.
• Product design/ideation/brainstorming - this is probably the easiest one to try if you don't have the time for a more elaborate one. This should, at a minimum, be a week of meetings with project leads, designers and even developers just brainstorming what you want to build. Coming up with new features. Output should be at least wireframes of major use cases. Although it doesn't let you test coding skills, it lets you see how pro-active the team is and definitely lets you check out their culture. And the output itself is useful (hopefully).

Do you know which one you will like before trying it out? Only if society would allow a nibble before a committed pick? :( 

The gist of the problem is very similar to the problem you face when you have to choose a chocolate from a  new box. They all look pretty good, but until you try them out you don't know for sure that you'll like what you've picked. Sadly it's socially quite unacceptable to nibble a chocolate first and pick it up only if you like the first nibble. But happily this is very much a possibility in the world of software outsourcing!  

You can stop reading here. You don't need to know why doing a small project is the correct answer - I'm that sure about this bit! And we have a lot of experience here, working on outsourced projects from all over the world for the past twelve years! 

But if you are interested here are some explanations.

Lets you check for fit with a small cost.

When you embark on a software project with an external team you just don't know if there is a good fit between your team and theirs. Do they understand what you say (even if they speak the same language)? Do they prioritize the way you do? Does their culture fit with yours? Does your team like their culture? Do they use the tools that you want them to use? Do they use them the way you expect them?

Endless questions which can never be answered satisfactorily without working together. Yet these questions are vitally important for the end product. Fit is everything. Let me say it again, the single most important thing for a successful outsourced software project is fit. But you only learn about the exact nature of the fit deeper down in an engagement. When it may be too late to exit if you suddenly realize there is not enough fit.

Lets you extrapolate

The results of a small project lets you extrapolate and make projection of what the result of the real (bigger) project will be. The way a team manages a small project is in essence the same as the way it will manage a bigger one. If the culture is "get it done" or "analysis paralysis" or worse "random walk" you spot it right away - and you can be sure that the same will apply to your real project.

With the results of the small test project you can make much better informed decision about a outsourcing partner and then decide to commit or not to commit.

Lets you exit early if you need to

Obviously. And exiting early is very important when you know something will not work and you need to start from the beginning to find a new partner. Ideally you should have the backup candidate ready, or even better, if you can afford it, make them do the small project in parallel - which is what I'm calling A/B testing loaning the word from webpage optimization.  

Lets you run A/B testing 

If you can afford it, get competing partners to do the same project in parallel (without telling them you are doing it!). This then let's you compare the result and find the better candidate.

Another variation is to get your internal team to do it in parallel and see how your own is different from the  new partner. If you do this, consider doing a presentation/retrospective session where the external provider explains their decisions to your team and check how the debates go between teams where their solutions differ. Very useful to find how the teams will work together (and separately your own secret review of your team's performance thrown in for free). 

Lets you fix issues either for this team or for your next one

There will always be some issues that need fixing when you are going to work with an external software team. The test phase lets you identify them right away, so that you can have that as a to-do list when you start working on the real thing. 

Even if you don't eventually decide not to go with the team, this list tells you what to watch out for when you look for when you go back to selecting another partner.

OK, here is my find from Dilbert today. Btw, if you don't like Dilbert strip then maybe there would be a fit issue with us! But if you do like them and you have a software project that you want to get done by the experts (and want it done at a lower cost), give us a knock - even better give us a test project! We may even do the test project for free to show off our prowess :)

Back from my shameless self promotion here is the Dilbert that I promised!