How much reliance should CFO’s and other finance professionals put on the Gartner Magic Quadrants for Cloud Financial Planning and Analysis Solutions and Cloud Financial Close Solutions?
Gartner is the most well-known and influential of all the analysts. You only need to scan Twitter and LinkedIn the week after the publication of a Magic Quadrant (MQ) to see for yourself. Every vendor represented in the report will put a triumphant message into a social media post or press release. But there is more to this than meets the eye, as one vendor employee described their inclusion in the Gartner MQ as ‘a necessary evil’.
And then there is the influence on potential purchasers of solutions. Within the reports, Gartner makes it abundantly clear that the Magic Quadrant is not supposed to be the only tool to use when creating a vendor shortlist. “It should be used as part of an organisation’s due diligence.”However, it is my experience that many organisations do not go beyond looking at the MQ graphic and the marketing messages.
Whether we like it or not, Gartner is a significant market influencer in every respect.
A research critique – the evaluation criteria
The fundamental question for me is whether it is possible to score a solution in any way without knowing the context in which people will use it. Let me use a car analogy here: if we were to plot car types/models using a Gartner style evaluation, I am sure something like a Tesla may appear top right. Will it be the right solution for my mother, who wants to drive to the shops every day and nothing more? Probably not.
You could argue that this is not how the user should use the MQ report, and they should not draw purchase decisions from it. Gartner states “application leaders should use the MQ as an aid to identifying the vendor most suited to support their financial operations”. But is this hard to argue when so much weight is on the vendor’s position in the MQ.
Let us look at the evaluation criteria in a little more detail.
Ability to Execute
There are seven criteria with defined weightings. As you would expect, the solution capabilities of product or service and customer experience are highly weighted. However, the overall financial health of the vendor and their ability to invest also has a high weighting. While I accept this is important, this would logically skew the MQ towards the largest vendors. Market responsiveness and record is also highly weighted, a vendor’s ability to respond, change direction. Now, this is where we would hope the smaller, more agile vendors will be able to bite back as the large vendors can be slower to react.
The last three criteria are interesting. Sales execution/pricing, marketing execution and operations, all of which have a medium weighting. As a prospective buyer, am I really interested in sales execution, or can I judge this myself during the sales cycle? Also, just because a vendor has a great marketing team does not mean its solution works. ‘Operations’ refers to a vendor’s ability to meet its goals and commitments. I am not at all sure what this means. Any measure would appear to be subjective.
By adopting this weighting, you can get some curious results. We will reference the 2019 FP&A report as an example with Oracle positioned as uppermost in the Leaders quadrant. “Oracle remains a Leader, due to its market traction with Oracle FCCS, its geographic, sales and product strategy, and its business model”. Reading further into the strengths and cautions, lowest quartile “for ease of implementation, ease of maintenance and upgrading, performance, and vendor satisfaction”. Two-thirds of reference customers would not recommend it. And “complaints from users of our inquiry service about SIs not keeping up to date with Oracle FCCS product releases“. Under the Gartner criteria, Oracle scored highly for its ability to execute.
Completeness of vision
This category has eight criteria with defined weightings. Those that are highly weighted include market understanding, product strategy and innovation, keeping an eye on the future needs of the market and customers. The other four criteria are based around a vendor’s strategy for marketing, sales, vertical/industry, geographical coverage and their business model (the validity and logic of a vendor’s underlying business proposition in this market). All these measures are subjective and seem to support the larger vendor organisations.
The above is my expert assessment, but it’s far from transparent, and I would ask Gartner to be transparent about the algorithm and the scores in each criterion. This would help consumers and consultancies alike. While this information remains unpublished, readers will continue to draw their own conclusions.
What is the right sample size for such an influential report? This is one point that has always niggled me. To quote from the Gartner report, “survey participants were reference customers nominated by each of the vendors in this Magic Quadrant”. Is this the right sample for an objective survey? I would suggest that a representative result should include a cross-section of customers – not only reference customers who are chosen by the vendor. It opens up the door to confirmation bias.
Gartner survey responses are only part of the equation, with Social Media Analysis also contributing to the final assessment, which makes the sample even more opaque, and it is not clear what factor influences the most.
The report is a global phenomenon, but it seems to have a significant US bias. This is a non-US point of view, but it is not clear. I would make this point to anyone who is looking at this report in the UK or Europe.
WANT TO GET THE LATEST DIGITAL FINANCE TRANSFORMATION INSIGHTS?
Sign up for our monthly newsletter and join 70,000 finance professionals all on a mission to make the use of digital practices the norm.
The Gartner approach to finance
It took me some years to get my mind around Corporate or Financial Performance Management (CPM/FPM). Gartner was one of the main drivers behind this concept and, up until 2017, published an MQ on this basis – Cloud Financial Corporate Performance Management. In 2018 Gartner officially retired ‘CPM’ and reclassified MQ’s into, ‘financial planning and analysis (FP&A)’ and, ‘financial close’. This may be to reflect two significant trends – increased focus on planning, and the emergence of a new category of solutions supporting the management of the financial close.
There is certainly a renewed vigour around planning and budgeting driven by the uncertain times we are facing and specific focus on this is no bad thing. I have also warmed to the financial close area, extending beyond the financial consolidation process into collaboration, reconciliation and disclosure.
However, this split may make the financial close MQ confusing to consumers, including vendors that are incomparable in terms of solution functionality, i.e. Workiva compared to OneStream. There also seems to be less of a focus on consolidation. This is where geographical influence is vital as financial consolidation requirements (e.g. multi-currency, complex investment, etcetera) are required less in the US than elsewhere in the world.
What I have seen in the market is buyers in two distinct camps. The first is the buyer with a best-of-breed strategy, an approach that is coming back into vogue (see my recent article). The second is those adopting a single-platform approach. As it stands, the Gartner MQs do not deal with the latter, other than the reader comparing the two MQs and seeing who appears in both.
The value of good research
A prospective purchaser can find significant value in the Gartner MQ reports. The detailed vendor analysis is insightful and contains useful information. However, not all vendors that may be of interest are necessarily included within the MQ, especially as new mid-tier cloud technology can challenge the legacy advance of the global large transaction players. We know of several solutions that are excluded.
If you are using Gartner MQs for a selection process, they are a reliable place to start.
However, we would recommend:
- Reading the Gartner report in full. It is worth an hour of your time to read end to end (most software vendors in the MQ can give you access to download it).
- Reviewing other analysts/sources with different methodologies, such as Forrester, BARC, G2 Cloud, Capterra and GenerationCFO.com (see below).
- Perform your own due diligence before making an investment decision.
The GenerationCFO approach
We adopt a slightly different approach to helping organisations identify the right solution for them.
We recognise that there is a wide range of technologies that can play a part in a digital finance transformation. CPM and financial close are just two, with analytics and Robotic Process Automation (RPA) playing an increasingly important role. We also recognise that it is not only the technology that needs to be right; the consultants that you work with are critical to success.
Our GenerationCFO.com Top 100 Showcase details technology vendors and consultancies that we think will be of interest to your finance teams and are being used by our community. It is being grown organically with a concentration on the UK at this stage. However, we think that this will eventually provide a source of information for organisations to search for potential solutions that meet their needs and criteria and potentially used by their finance peers.
If you would like to be considered for inclusion in the ‘Top 100 Showcase’ or you are included and would like to claim your listing so that you can update the details, please contact us at email@example.com.
The Gartner Magic Quadrants for Financial Planning and Analysis has recently been released. The Financial Close MQ is due before Christmas. Watch out for my thoughts on the FP&A report in the coming days.
Gartner (Robert Anderson, John Van Decker, Greg Leiter), “Magic Quadrant for Cloud Financial Planning and Analysis Solutions”, 8 August 2019.
The post The Gartner effect: are magic quadrants worth the hype? appeared first on Generation CFO GENCFO.