Why Is Excel So Hard To Kill?

The first is that strategic systems for certain tasks are simply not prioritised by central IT functions, meaning that some groups of users may feel they are forced into a situation where they must take action themselves. Such examples can happen in regulatory reporting where an external deadline exists, but insufficient IT budget is available to buy or build a strategic solution. In these cases, the near-ubiquity of Excel makes it a popular tool among users, because virtually everyone has it available, even in the most locked-down IT estates, making it a useful and interoperable tool.

Secondly, because it is so common, the barrier to entry is typically very low – there are rarely any software requests to fill in, since the Office suite is pre-installed for nearly all users, meaning that not only is it readily available, it’s also a tool with which all users are already highly familiar. Whereas you may need to train a new user how to access a database, it is unusual that you find a user in a modern organisation who cannot use the core functionality of Excel.

The lack of other tools also plays a role here. In many organisations, Excel has become the “go to” tool for a lot of users, meaning that it is used for a variety of tasks well outside what it was originally designed to do, and extended with Visual Basic macros to perform a wide range of functions. Indeed if you dig a little, you’ll discover that Excel has been (ab)used for activities as diverse as a 3D rendering engine, a cross-stitch pattern generator, a flight simulator and (one of my personal favourites) an engine for unsupervised machine learning. Whilst this is an amazing display of human ingenuity, it is not necessarily the most efficient way to operate a business, and there are now a wide range of tools that can be used to perform specialised functions more effectively.

Ownership of processes can also play a role in the continued use of Excel. Individuals may have built up a lot of expertise around how to complete a particular report and may feel a loss of control, or even that their job is at risk if these tasks are moved out to an IT solution. Moreover in situations where requirements are still evolving, the easy to prototype nature of Excel means that end users can rapidly adjust and alter their systems to keep aligned with new requirements, often without an obvious direct cost (that there is no cost is of course a fallacy, but it can be extremely hard to quantify)

Finally tactical-thinking can play a part in the continued re-emergence of Excel as a solution. This is really a tendency to see all the benefits of the four points above, and to overlook the challenges, but in many organisations, this can be a significant cultural problem. It is often slow and difficult to get a strategic IT solution developed, and in some cases the very people who are needed to push for that may themselves feel threatened by it. It is therefore entirely unsurprising that some simply shy away from it entirely and engage in tactical thinking where they promote the ability to develop new reporting “for free” by using existing subject matter experts and asking them to build an Excel solution.

So if these are the drivers for Excel popping up time and time again, what’s the solution?

To start with I’d advocate normalising a wider range of tools, that end-users can use to develop prototype solutions. You might think that the last thing an organisation with too much Excel needs is MORE tools for end user computing, but hear me out…

By bringing in new tools, there’s an opportunity to implement them in a more controlled manner. Excel functionality has crept over decades to provide the capabilities it has today, and few organisations would have chosen to implement it in the way they have if it had first landed with the capabilities that exist today, set against today’s regulatory and governance backdrop. However, as it has crept, attempts to control the spread of poorly governed end user computing have been at best a rearguard action.

Modern low code / no code tools have been designed with a little more governance in mind and can be implemented in a more controlled way, both in terms of change governance, and data governance.

For reporting processes I would advocate use of tools like Power BI for the charts and visualisations, and tools such as Power Automate and Power Apps to build low code, low cost but more controlled processes and workflows.

Taking these in turn, Power BI is designed from the ground up to integrate with user access via Active Directory, meaning that only groups of users who are authorised to view dashboards can do so. Furthermore, with no access to the underlying data for standard report users, there’s no danger of accidentally overtyping data and no ability to simply attach the file to an email and send data outside the organisation (maliciously or accidentally). Power users who will create reports have additional access rights and can design reports, but these are then published before being made available to users, making for a natural control point in the process, and also ensuring that there’s only one live version of the report at any one time. Finally, data engineers will have access to the underlying database (as a Microsoft Partner we’d advocate Azure SQL or Azure Synapse for most use-cases, although of course many database solutions are available), which should be populated using automated processes which are themselves under Application Lifecycle Management control (for Microsoft users this might be Github or Azure DevOps).

For processes that need “screens” to complete activities or fill in data, Power Apps is a good solution. It can also be configured to work with source code control via Azure DevOps (albeit this integration is still in beta, but will likely be part of the core platform before the end of 2022), meaning that revisions to the screens and their logic are subject to a process that publishes them to the live environment. Again, this is an opportunity to ensure adequate testing / peer review / IT sign-off etc has been obtained

Finally the functionality to update, sign-off and send reports out can be encapsulated within Power Automate, with standard processes (again themselves under source code control) that can require authorisation to apply amendments or edits, or to release the data to a regulator or end-users.

These tools can be deployed in a manner that suits any given organisation’s appetite for end-user computing. In some instances, the gateway between development and live versions of software is controlled within the IT division, in others it may require simply an appropriately senior (and skilled!) sign-off within the business. However, what they do is to democratise the organisation’s software development capability, whilst avoiding many of the pitfalls of Excel, such as lack of source code control, and data security risks.

There is no silver bullet for dealing with the governance challenges of end-user computing neither denying the issue, nor embracing yet more Excel solutions are viable solutions. Cultural change to recognise that there are legitimate use cases for end user computing and empowering teams to develop them in a safe and controlled environment is the first step towards a more joined up and better governed data architecture.