To evaluate vendors or not, that is the question*.

Pt1: What are you looking for when buying tech?

[*If you’re professionally involved in procuring tech, I have a few more questions for you too…]
Author: Matt Tett (Enex TestLab) March 2022

OK, I admit I have an odd obsession with tech. And mine’s maybe a bit weirder than most, because our teams make a living out of putting the latest so-called advancements (or iterations or whatever you want to call them) to the test.
But I know I’m not alone in being just a little bit cynical about vendors’ claims.
So, I sympathise if you’re someone who works in or with technology contracts and procurement. The expectations put on you are ridiculous.
You’re very often the piggy in the middle between what the organisation requires and what the vendors offer to deliver. Somehow, you’re expected to perform a magical balancing act of matching scope of requirements with statements of work.
Before I get into how I think your job can be made easier, here’s a series of questions for you, if I may?

• How much does your organisation spend on projects involving technology each year?
• Is technology seen as a cost or an investment?
• How many of those technology projects deliver 100% of expectations? (And how many fail?)
• If your organisation conducts proofs of concepts or pilots of the technology are multiple short-listed vendors involved ?
• Do your purchasing decisions come from paper-based audits of submissions in response to RFQ or RFTs?
• Have you ever heard a vendor say anything negative about the product or solution they are offering?
• Do you want to talk about ensuring the value of technology is measured and delivered?

Yes, that’s a lot of questions. Let’s get down to some opinionated answers:

Q. How much does your organisation spend on projects involving technology each year?

It might start in the tens of thousands or well into the millions. The starting point is just that, because as you know too well, tech budgets can quickly blow out – particularly when you factor in additional expenditure like scope-creep, over-run, and initial investments falling short of expectations which then require remediation or gap filling.
And is the total cost of ownership for the lifespan of the technology factored in too? Surprisingly, a lot of organisations separate the capital expenditure from the operating budget and never the twain shall meet, let alone consider the roll off, decommissioning and disposal when products reach the end of their lives.

Q. Is technology seen as a cost or an investment?
I strongly believe technology can only be counted as an investment if it delivers on requirements such as increased productivity or other efficiencies. Otherwise it’s a cost. If it fails, the cost is an outright loss – and that’s just the start, because a loss has a ripple effect causing further impact on the organisation.

Q. How many of those technology projects deliver 100% of expectations? (And how many fail?)
Call me cynical, but I suspect very few technology projects ever deserve to score 100%. I don’t even want to guess how many issues are hidden behind some of the technology success stories we hear about in the media or through our networks.
And it’s not even sarcastic to suggest more projects than are publicly known deliver less than desirable outcomes when compared to the original requirements and expectations.
But there is another way to answer this – and I’ll get into that soon. Here’s a hint: technology projects are much more likely to stay on track with benchmarking and measurement of value throughout the project lifecycle.

Q. If your organisation conducts proofs of concepts or pilots of the technology are multiple short-listed vendors involved?
I really want the answer to be “Yes”. But I know from decades of experience most purchasing decisions are still based on vendor submissions and their claims, even if they are short-listed.
Too often, even when an organisation progresses to proof of concept or pilot, it’s only with a single vendor’s product or solution. Frankly, it’s kind of difficult to make a fair comparison between how number two or three, or six or seven, may have fared.

Q. Do your purchasing decisions come from paper-based audits of submissions in response to RFQ or RFTs?
Don’t get me wrong, relationships are key and I do love vendors. I know desktop audits of paperwork and RFQ and RFT responses play a valid part in short-listing, because it’s important to check each submission meets your compliance, capability, financial and commercial needs.
But when you get down to a short-list, what do paper-based audits tell you about how the products stack up technically?
And you can’t judge on price because price doesn’t necessarily equate to value!

Q. Have you ever heard a vendor say anything negative about the product or solution they are offering?
Ha! As if anyone has ever heard vendors say anything but what their business development and marketing teams want you to hear. Let’s face it, they have their widgets to push so you’re unlikely to hear them stating anything that doesn’t reflect favourably on their products.
We’ve never seen a vendor’s response or marketing material mention the cons and shortcomings of their widgets, only the pros and all the good things they promise to deliver in spades.
So how can you find out if the claims a vendor makes in an RFQ/T response are true? And more importantly, how can you find out if their widget will meet your requirements?
I’m so glad you asked.

Q. Do you want to talk about ensuring the value of technology is measured and delivered?
I’m always happy to talk about the work we do at Enex TestLab.

We’re often asked to be the independent piggy in the middle, sitting between project stakeholders and sometimes groups of vendors to verify claims and provide evidence-based reports to help stakeholders make more informed decisions.

We have more than three decades’ experience evaluating vendors’ tech products for clients ranging from government and corporate to private clients, publications and even vendors themselves. So, we’ve seen and heard it all when it comes to measuring the real-world value of technology – and we’re known for our truly independent, unbiased and educated opinions.

What more? Episode 2 Next Week.