Evaluating Evaluations

Your company needs a new IT solution; let’s say a new 3D design, a CRM, or a fully new business system (ERP).

What do you do? Well, you setup a project team that starts collecting the business requirements, scans the market for potential solution providers, writes and sends out Requests for Information (RFI), and presents a list of suitable solution providers to the evaluation projects’ steering committee.

Then you give the team members permission for the fun part: they can travel around to visit reference companies and make the providers do product demonstrations where everybody can shine by asking smart questions.

Typically these evaluation activities come to a conclusion when you issue the formal Request for Proposal (RFP) to the most promising providers. This RFP is a first step to nail scope, time, and commercial issues and prepare for the final selection and contract negotiation.

So, you probably end up, after a lot of ranking and weighting different evaluation criteria, with two short-listed solutions. It is smart to do at least with one of them some kind of proof of concept activities. This to make sure the solution and provider perform as expected and to gain a firm basis for the actual contract negotiations.
If the proof of concept succeeds, you negotiate the final contract and start the project that will deliver your new 3D design, CRM, or ERP solution.

Not so much wrong with an evaluation process like this. But some basic questions do arise:

• Is your evaluation process in line with how your company makes decisions?
• Will a large excel sheet with weighted factors showing at the bottom solution A with 5.6 points and solution B with 6.3 points make you more comfortable to pick solution B?
• Will the senior management trust your findings or anyway follow their gut feeling?
• Does your evaluation process ensure buy-in of all stakeholders?
• What will you do if some parts of the organization demand solution A and reject B flatly?
• Is your company mature enough or really ready to start implementing the solution after signing the contract?
• Are you sure you’re asking for the right solution?
• Is your FRI/RFP material flexible enough to give room for the provider to bring up their best proposals?
• Are you sure the required partnership with your provider to deliver the solution successfully will be in place after the evaluation process?

When you go out looking for a new solution consider the questions above before nailing your evaluation process, and most importantly:

1. Make sure you agree beforehand with the senior management based on which facts & feelings they want to make the actual decision/selection.
2. Consider using the know-how and expertise of your potential providers early on when defining your requirements.
3. Maintain a win-win relation with your provider while negotiating solid terms & conditions.