A/B Testing the culture of your company

Wouldn’t it be great if you could A/B test the policies that you take for your company just like you A/B test user adoption for your designs? If we could somehow try our decisions out and then take the one that works better it would take the heartache out of a lot of everyday decisions.

Let’s face it, the way we run our companies, and I’m talking about everyone - from the mini startup to the mega corp., is purely a version of gambling. Let me break down some of the common “rationale” for our policies for the company -

Excuses for bad policies:

“That’s the way it’s done everywhere”

And probably that’s why most companies have dismal work cultures.

“If we don’t do this now, things will go out of control”

As if it’s a kindergarten we are running or worse a prison :(

“There must be checks and balances”

Yeah, but when you have to say it out to justify a policy chances are you are forcing something in that’s no check and no balance.

Anyway, you get the picture (and this is just a rant post). Getting back to my point, A/B Testing - most of the times we are never sure if the policy we are putting in place is a good policy that makes the company efficient and makes the people happy (yes, you can have them both!) or if it will just sap the soul out of the people and lead to inefficiencies and backstabbing. Take the classic “Performance Review” the classic “That’s the way it’s done everywhere” thing. Is it good or bad? Without a performance review your best people will not feel they are being prized or valued. But with performance review aren’t you bringing in competitiveness? Particularly in software development doesn’t the best coder in the world go through a patch of really bad performance (which is offset by 10X performance at other times)? There is no solid gold answer - only “buts” and “ifs” and “depends”. btw if I’ve got you riled up here’s a good read from HBR - Why More and More Companies Are Ditching Performance Ratings

So if we just had a simple way of trying out a policy (maybe in part of the company) and trying something different (with another part) and the checking for the results and pick the one that’s better. Anyway without a foolproof system for A/B testing here’s what we do at Kaz whenever we we trying to put a new policy in place or change an existing one:

Have a “feel out” phase

Every policy we adopt goes through a “feel out” phase where it’s discussed formally and informally during team meetings, lunches even hallway discussions. This gives early feedback about the policy, get amazing ideas of what might work and what might not work and works as great PR campaign for the policy.

Introduce incrementally

All policies are introduced very slowly and in parts. This makes it easy for us to find our mistakes and fix them without making a huge mess. It’s amazing how many times we have fallen face in even when the plan sounded great during our feel out.

Seek feedback

This is the important one - seek feedback from the people who are most affected by the change. There might be some very good ideas about tweaks that might work.

Be prepared to change or roll back

Nothing should be set in stone. As long as we achieve the goal we should not bother about how we achieve it. So if something doesn’t work just change it or roll back. Remember that it’s not a prison that you are running and there is absolutely nothing wrong for a company to admit that it was wrong.

Measure company culture

Now this is hard one. If you can’t measure and put numbers on culture you can’t find out if something is working better or making things worse. But it’s not easy and some would say next to impossible. Our thinking is that you don’t really need perfect system (there is none), as long as you have some measure it’s better than none. So a simple poll might be a good way out, or you can go the full Titanium card path and try something like MIT’s sociometrics (MIT being MIT had to come up with device to measure culture):

We wrote a series sometime back on how we do and did things in quantifying culture The method doesn’t matter as long as you have some way of understanding the effects that you policy changes are causing.

But still, we would love a scheme where we could do those A/B tests easily. Maybe MIT will come up with something? :)