Keys for Improving Predictive Analytics in Finance

Keys for Improving Predictive Analytics in Finance

November 20, 2019 DATAcated Challenge 0

Many companies spend months on budgeting processes only to have the budget become less meaningful as the new year begins and then the focus is about forecasting and budget changes. The directional outputs become talking points instead of actionable insights. In one case I was working on budgeting for fee income and the high-level, top-down, blended growth goals averaged 20%. When it came time to do the bottom-up detail work there was a disconnect in the necessary revenue growth. The bottom-up required an organizational growth challenge of 70% vs. the 20%. This was discovered three months into the budgeting process and at this point leadership wasn’t going to accept an explanation that we might not have done enough research and analysis on the front-end. Applying a predictive analytics approach would likely have greatly assisted in identifying and narrowing or eliminating this gap. In another example, I had a group of stakeholders eager to drive the sales budgeting to sales rep, customer, market segment making them more accountable for account growth. The model was so complex that the investment of time and energy to maintain and update it would not have translated into significantly better outcomes. I commended them for their proactive thinking. We spent two weeks collaborating on a new model that leveraged a what-if modeling approach. The model was an improvement on the existing process, it was repeatable and simple to use, but dynamic enough with scenario levers and historical context and insights on goals and potential challenges.

Retrospective insights / analytics, which I am using to cover descriptive and diagnostic activities, are strategically important for organizational management as well as remaining transparent and accountable to internal and external stakeholders. By reviewing what has happened organizations understand the “why” based on empirical conclusions. This helps an organization to be more effective in inventory control, staffing, selling, etc. For the most part conclusions are restricted to previous decisions and actions. The process for retrospective insights can take a long time to complete and change management and action may be delayed due to politics and “fire-drills” / conflicting priorities and accountabilities. Retrospective insights also typically lack a “what-if” component. Here is an example:

As I reflect on my life and career path one of my more influential experiences occurred in late 2015 when my wife and I were having dinner with a close family friend who was pursuing a Ph.D. in nursing informatics. It was obvious she had a passion for the topic and our discussion was engaging and informative. We ended up discussing opportunities in the practices of data science. My friend pointed me to some very informative articles and resources about the explosion of needs for technically oriented, smart, resourceful, adaptable, innovative and learning motivated people who would be able to step into a wide variety of roles ranging from data analysis & analytics to data science. Many of the roles in data science and analytics require additional skills and experience that might not have been the case in finance roles. My wheels began to turn. I did my own research and decided to enroll in the big data certificate at the University of St. Thomas (after meeting with professors to do my own requirements gathering and analytics). My decision to enrolled and the ensuing experience has transformed how I viewed my career, skills and experiences along with the challenges and opportunities in finance roles.

Over the last five years I’ve read my fair share of articles and books on data analytics and data science, with a lot of focus on big data. I’ve also attended numerous events and connected with a wide range of people to develop my knowledge and evolve my own thoughts across a wide variety of topics that I group into data science and analytics. I have read about the expected growth in demand for people in data science and analytics roles continues to demonstrate an oversized requirement in comparison to the available talent. And more recently, according to a Forbes article written by Louis Columbus (May 13, 2017, 09:21pm) “59% of all Data Science and Analytics (DSA) job demand is in Finance and Insurance, Professional Services, and IT”. And according to PWC “The 2020 estimate calls for 2.7 million job postings for data science and analytics roles” with data engineering and data science leading the charge with growth rates of between approx. 37% to 43% (

It seems that people in finance focused roles, e.g. traditional FP&A, Finance Management, etc., might not want to learn (or haven’t needed to learn) the more technical skills of a data analyst or scientist, but that doesn’t mean there aren’t significant skill development opportunities and synergies that will greatly leverage people in finance to move beyond descriptive (retrospective) analytics.

As a result of years in a variety of finance and analytics roles, I’ve increasingly realized that most of my past work experience in finance roles was already relevant to the types of projects and analysis that organizations could tackle with analytics and data science and take to the next level to realize and drive more proactive value in the organization. Whether it be more robust and dynamic forecasting models, improved visibility to expectations and margins, deeper insights on segmentation and pricing, reduced process times (repeatability and simplicity), improved customer service, etc., the insights generated through modeling provides organizations more proactive outcomes. Tom Davenport, in his book big data @ work, points out that “the financial services industry was perhaps the first to adopt big data.” He goes on to state “it is likely that in the near future, big data will find its way into corporate finance departments.” I’d argue the time is already here. Finance professionals have diverse knowledge about the business, they maintain numerous cross functional relationships, they are stewards of business assets, they help set and manage strategic direction, they support capital management, analyze acquisitions, mergers, new business opportunities, plant closings & relocations, business shutdowns, etc.

There’s really no reason for organizations or finance hiring managers to not see the value drivers for technical skills more often attached to analytics, statistics, and data science. Microsoft Excel is so embedded in the world of business that it’s the go to tool for a multitude on applications. The wizardry that can be accomplished with Excel is truly amazing. Apply visual basic coding, macros, and formulas and nearly any numeric model can be built into a theoretically repeatable and maintainable process. Layer on top of this the multitude of applications ranging from Online Analytical Processing Applications like Hyperion Essbase, Cognos, MicroStrategy, OBIEE, etc. to others calling themselves something different, think elastic cubes and in-memory technology (nothing new here). The finance toolbox is vast. When you think about the top skills for finance roles. Robert Half’s opinion is that Analytic Skills are number 5 and the ability to communicate is number 3 (source:, May 12, 2018 ). Robert Half says, “candidates looking for a successful career in finance must demonstrate their analysis abilities with real-world examples and KPI driven results”, but they didn’t even consider the fact so much focus has been on descriptive analytics. IBM published the following statement on October 22, 2015 “With the increasing role and responsibilities of the CFO, financial professionals seek solutions to help provide answers these questions, and drive performance across the enterprise. Today, predictive analytics are changing the game for companies and their executive teams.” ( And according to InTheBlack “…[finance managers] want someone who can look at big sets of data, pull reports that add value and present them to the business’” (Nicola Heath, Jan 2, 2018).

To do this kind of work traditional tools that finance professionals have relied on will be overwhelmed by the volume, velocity and increasingly more unstructured nature of data coming from the increasing number of diverse data sources, especially those that aren’t residing in relational databases. Excel currently supports 16,384 columns and 1,048,576 rows in 3 sheets (that’s just over 51.5 billion cells of data), and perhaps more sheets where there is enough memory. That’s impressive, but performance of Excel on large datasets becomes unstable and repeatable and ease of use quickly disappear. For example, what happens when one has a dataset of 157 million records across even just five to seven dimensions with one measure? Nothing, Excel becomes irrelevant. Furthermore, when it comes to unstructured data Excel lacks the capabilities needed to perform rapid and intuitive analytics. Finance departments need to be strategic about data, think about data integrity, and governance along with process efficiencies, automation / scheduling, and an ability to leverage both descriptive and prescriptive capabilities is crucial to the health and competitive position of a business. One can pick a number of tools and just because it may be the less expensive option doesn’t mean it’s going to be a good strategic selection.

The time for change is now, hiring managers must act to drive and foster their finance staff in the engagement of growing their technical skills to look beyond describing what has happened (simple trending) and really push into projecting into the future, engage the business in ongoing conversations, have constructive conversations about what drives the business, and to pull the actionable insights together into constructive and easy to understand business language and interactive models / visualizations. Some typical predictive approaches are regression analysis, both simple and multiple regression are leveraged to identify business drivers and focus business efforts on managing to those drivers. While time series analysis can be used to produce more accurate projections based on how those drivers are expected to change. Credit scoring allows for an assessment of a borrower’s credit worthiness. Finance management can develop a staff that truly predicts outcomes instead of looking only at the past and assuming the future will be the same. To this point an article published on by Larry Maisel (Jul 9, 2017) identified five keys for applying predictive analytics in FP&A. “According to the 2017 CFO IT Survey, over 70% of finance executives said they plan to substantially increase the use of data analytics in the next two years, to support decision-making and improve business partnering. Fully 68% of respondents plan to improve their data analytics skills in the coming year.”

One big barrier for finance success in partnering with IT is that they commonly speak a different language, which means crafting a bridge between a desired end state and the nuts and bolts. Finance and IT can start by agreeing that cost is important but also realize that one must be aware of technical requirements, the current environment, what is needed for the end state, ongoing costs, upgrade issues, ongoing maintenance and support, etc. Executive leadership must be the champion of transformation on how the departments partner and communicate. Embracing a mutually collaborative approach where both sides have a key subject matter expert representative within each domain is one method to take. I’ve seen a few successfully examples, but not many and a lot of my conversations have confirmed that there are common big roadblocks. I recall an example where the business line had identified a key metrics initiative need. They had done the hard work on proof of concept, user story construction and key metrics identification. However, they neglected to involve IT in the POC to full gather the technical requirements to ensure a plug-and-play compatibility from end-to-end. The project couldn’t successfully move forward without key technology constraints being resolved. At this point the organization had already spent three months of time and money, including high priced outside consultants, which was going to sit idle until IT could catch up. Due to timing issues, however, the project deliverable was not going to be achievable in time for the ensuing business cycle so an entire year would lapse before implementation. This was a large opportunity gap and clearly painted a picture of lack credibility. The alternative course would have leveraged IT experts at the beginning of the process to help complete the investigation on the technical requirements, including a working model to prove end-to-end execution success.

The pace of change and innovation continues to accelerate. IoT, artificial intelligence, machine learning, robotic process automation, and numerous other transformations will challenge finance and IT to partner more effectively and discover options to share knowledge. Even in a world where we hope to embrace the idea of self-service analytics the need for transparency and communication will only heighten as organizations find themselves being challenged by a diverse universe of hungry, educated, and determined entrepreneurs who will drive innovation even faster.

Tripp Parker

Insights, Actions and Empowerment

Leave a Reply

Your email address will not be published. Required fields are marked *