The world is awash with informal systems, sometimes called “shadow IT.” The myriad of small informal systems delivers enormous value to almost every organization known to man. Informal systems are typically based on software tools such as Microsoft Office 365 that includes Excel and Access, Google G Suite or Apple iWork that includes a spreadsheet and Apple FileMaker.
Informal systems are typically created in an ad hoc manner to cover off gaps in the application portfolio of formal systems that most organizations operate.
A subset of examples of informal systems includes tracking work status, forecasting cash requirements, tracking product returns, illuminating data inconsistencies between two systems, calculating bonuses, analyzing survey results, scheduling equipment, analyzing inventory positions.
The appeal of informal systems includes:
- Low cost for software tools and development effort.
- Ability to start extremely small and allow the system to evolve organically.
- No need for planning.
- No need for requirements gathering.
- No interaction with the IT department.
- Incredible malleability.
- Quick responsiveness to changing requirements.
- Concurrent operation and development.
- Avoid rigor of the systems development process.
All these appealing characteristics of informal systems come with risks that are often not recognized and sometimes deliberately ignored.
Below are some of the common risks that come into play as informal systems grow and even become mission critical.
Restricted to one end-user
Informal systems are based on a filesystem as opposed to a database management system (DBMS). This restricts their use to one end-user at a time in many situations.
This one end-user limitation risks the system becoming overwhelmed as the data volume grows and the value of the informal system increases.
Restricted in data volume
Informal systems are restricted in the maximum number of rows that the underlying filesystem can manage.
This limitation leads to the risk of non-performance as the volume of data approaches what the filesystem, memory, disk speed and processing power of the workstation can handle.
Difficult software debugging
The development tools used to build informal systems are quite primitive. This limitation makes finding software defects quite difficult and time-consuming.
As the functional complexity of informal systems grows, the risk that the software will become increasingly unstable and the output will become unreliable also grows.
Informal systems tend to crash quite frequently. Most crashes are caused by the informal software encountering a condition in the data that was not anticipated by the designer who is typically not a software developer.
The associated restarts and sometimes reboots of the workstation will risk the confidence that the organization has in the output of the informal system.
Knowledge loss on staff turnover
Informal systems tend to be the creation of one bright, energetic business person.
When that person is promoted, head-hunted, transferred, laid off or fired, the risk is that the detailed knowledge of the architecture and operation of the informal system will go with them.
Inadequate data validation
Informal systems tend to focus on minimizing the data entry effort of the end-user entering new data. This focus often means the data is not being validated very carefully by the informal system.
The resultant risk of data quality lapses will gradually but inevitably undermine the confidence that the organization has in the output of the informal system.
Inadequate data integrity checking
Initially, informal systems tend to focus on data for one entity such as customers or orders. As the demand for functionality grows, additional entities are added. This situation creates the potential for customers without orders or orders without customers. This situation typically results from inadequate data integrity checking.
The risk of poor data integrity is the increasing number of system crashes and reports whose content will become increasingly unreliable.
No real-time capability
Informal systems largely operate in batch mode. Data entry is usually real-time but outputs such as queries, charts, and reports are batch operations. Even worse is that while a batch operation is running, no other batch operation or data entry can be performed.
The absence of real-time capability typically leads to the implementation of a scheduler function to manage the growing batch reporting workload that will execute overnight. This approach risks becoming overwhelmed as the number and frequency of report requests grows.
Mitigating the risk of informal systems
When these informal systems risks turn into reality, the management response should consist of the following:
- Be grateful for the enormous value that the informal system has delivered for your organization over many years at an incredibly low cost.
- Be grateful for how the informal system has increased your organization’s understanding of the detailed requirements of the application domain being managed by the informal system.
- Recognize that it’s time to replace the informal system with a formal system that will address all the risks and realities you’ve encountered.
- Remind yourself and everyone else that the higher cost of the formal system reduces many risks, positions the application for huge future growth and still delivers an exciting net benefit to the organization.
How would you make the case to management to replace an informal system that is operating with serious risks well beyond its best-before date? Let us know in the comments below.