Shining light on shadow IT: managing risk around informal systems

The world is awash with informal systems, sometimes called “shadow IT.” The myriad of small informal systems delivers enormous value to almost every organization known to man. Informal systems are typically based on software tools such as Microsoft Office 365 that includes Excel and Access, Google G Suite or Apple iWork that includes a spreadsheet and Apple FileMaker.

Informal systems are typically created in an ad hoc manner to cover off gaps in the application portfolio of formal systems that most organizations operate.

A subset of examples of informal systems includes tracking work status, forecasting cash requirements, tracking product returns, illuminating data inconsistencies between two systems, calculating bonuses, analyzing survey results, scheduling equipment, analyzing inventory positions.

The appeal of informal systems includes:

  1. Low cost for software tools and development effort.
  2. Ability to start extremely small and allow the system to evolve organically.
  3. No need for planning.
  4. No need for requirements gathering.
  5. No interaction with the IT department.
  6. Incredible malleability.
  7. Quick responsiveness to changing requirements.
  8. Concurrent operation and development.
  9. Avoid rigor of the systems development process.

All these appealing characteristics of informal systems come with risks that are often not recognized and sometimes deliberately ignored.
Below are some of the common risks that come into play as informal systems grow and even become mission critical.

Restricted to one end-user

Informal systems are based on a filesystem as opposed to a database management system (DBMS). This restricts their use to one end-user at a time in many situations.

This one end-user limitation risks the system becoming overwhelmed as the data volume grows and the value of the informal system increases.

Restricted in data volume

Informal systems are restricted in the maximum number of rows that the underlying filesystem can manage.

This limitation leads to the risk of non-performance as the volume of data approaches what the filesystem, memory, disk speed and processing power of the workstation can handle.

Difficult software debugging

The development tools used to build informal systems are quite primitive. This limitation makes finding software defects quite difficult and time-consuming.

As the functional complexity of informal systems grows, the risk that the software will become increasingly unstable and the output will become unreliable also grows.

Frequent crashes

Informal systems tend to crash quite frequently. Most crashes are caused by the informal software encountering a condition in the data that was not anticipated by the designer who is typically not a software developer.

The associated restarts and sometimes reboots of the workstation will risk the confidence that the organization has in the output of the informal system.

Knowledge loss on staff turnover

Informal systems tend to be the creation of one bright, energetic business person.

When that person is promoted, head-hunted, transferred, laid off or fired, the risk is that the detailed knowledge of the architecture and operation of the informal system will go with them.

Inadequate data validation

Informal systems tend to focus on minimizing the data entry effort of the end-user entering new data. This focus often means the data is not being validated very carefully by the informal system.

The resultant risk of data quality lapses will gradually but inevitably undermine the confidence that the organization has in the output of the informal system.

Inadequate data integrity checking

Initially, informal systems tend to focus on data for one entity such as customers or orders. As the demand for functionality grows, additional entities are added. This situation creates the potential for customers without orders or orders without customers. This situation typically results from inadequate data integrity checking.

The risk of poor data integrity is the increasing number of system crashes and reports whose content will become increasingly unreliable.

No real-time capability

Informal systems largely operate in batch mode. Data entry is usually real-time but outputs such as queries, charts, and reports are batch operations. Even worse is that while a batch operation is running, no other batch operation or data entry can be performed.

The absence of real-time capability typically leads to the implementation of a scheduler function to manage the growing batch reporting workload that will execute overnight. This approach risks becoming overwhelmed as the number and frequency of report requests grows.

Mitigating the risk of informal systems

When these informal systems risks turn into reality, the management response should consist of the following:

  1. Be grateful for the enormous value that the informal system has delivered for your organization over many years at an incredibly low cost.
  2. Be grateful for how the informal system has increased your organization’s understanding of the detailed requirements of the application domain being managed by the informal system.
  3. Recognize that it’s time to replace the informal system with a formal system that will address all the risks and realities you’ve encountered.
  4. Remind yourself and everyone else that the higher cost of the formal system reduces many risks, positions the application for huge future growth and still delivers an exciting net benefit to the organization.

How would you make the case to management to replace an informal system that is operating with serious risks well beyond its best-before date? Let us know in the comments below.

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada
Yogi Schulz
Yogi Schulz
Yogi Schulz has over 40 years of Information Technology experience in various industries. Yogi works extensively in the petroleum industry to select and implement financial, production revenue accounting, land & contracts, and geotechnical systems. He manages projects that arise from changes in business requirements, from the need to leverage technology opportunities and from mergers. His specialties include IT strategy, web strategy, and systems project management.

Featured Download

IT World Canada in your inbox

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.

Latest Blogs

Senior Contributor Spotlight