Evaluating technology suppliers has long been conducted through the RFP process but is this really the best way to evaluate one technology supplier against another? We have run dozens of these processes over the last few years for a range of technology solutions including eCommerce, CRM, Warehouse Management, Content Management, Customer Services, 3PL, Analytics and so on, using a process whereby we deep dive into the functional and non- functional requirements for the solution and evaluate vendors’ responses against our selection criteria. We go through stages narrowing down suppliers, conducting negotiation continuously to ensure best price and service provision until one remains victorious and wins the project.
The problem, however, is that in certain domains, most notably eCommerce we come across the same 4/5 vendors every time, their solutions are mature and the first stages of the RFP process actually tells us nothing we didn’t already know. For example functional matches (out of the box) requirements range between 70-80%; costs for license, hosting, support and implementation are within a close range and independent analysts rate them all as leaders. I recently had to supply a business case for eCommerce re-platforming to the CEO of one of our customers and we estimated the likely 3 year TCO across the 5 major players without any direct engagement. Guess what, after we received the RFP responses our margin of error across all five was within 10%.
Our customer had spent two months working with us to define requirements, produce the RFP collateral and then another 6 weeks engaged in an RFP process to get the first round responses. The big issue is that now all the major vendors offer similar functionality, delivered in a similar manner (or so it appears on paper) and for a very similar price. The RFP only serves to illustrate just how similar the vendor offerings are and waste a significant amount of time and money verifying what you should already know assuming you’ve got the right consulting specialist onboard. What RFP’s are good for is formalising a negotiation process, testing a vendor’s willingness to engage and reviewing how committed they and their partners are to you and the project.
Rather than spend 2-3 months in a process where you learn very little we think the RFP process should be re-engineered. What about starting by asking vendors to commit to a time boxed Proof of Concept using their standard product to meet a number of scenarios relevant to your project? We could take one functional area of the solution or a number of features, user journeys etc and ask the vendors to show us how they would deliver this into a production ready environment. What we should be evaluating is how quickly those features can be delivered using standard functionality, where the configuration and customisation effort is and how long it took to do. These activities need to be tracked in project management tooling that provides the customer complete visibility (requirements, designs, functional scope, estimations, development burn down) on what tasks were required how much effort was associated to them along-side demonstrable software. In this way we mix real world scenarios with delivery methodology and the end product which is a practical assessment of vendor capability rather than a paper based exercise. Of course for this approach to work you need a consulting partner that works extensively in the space with deep experience of the various technologies and you still need high level scope in order to provide TCO comparisons. But by using a POC as the primary evaluation tool you will learn much more and might even come out of this process with workable software – now wouldn’t that be something?
By Mark Adams