Insights
In today's world of digital transformation, enterprises find themselves transforming their business processes and workflows from paper-based manual labor to computerized and automated systems. While it is only a natural and healthy trend, it requires long-term vision, planning, and continuous iterations to guarantee success.
The Problem
A large number of organizations use digital solutions for enterprise resource planning (ERP), customer relationship management (CRM), product lifecycle management (PLM), and others as a system of record (SOR). Companies like SAP, Microsoft, and Salesforce sell these products as a packaged solution or software-as-a-service (SaaS). Business units inside these organizations may use additional, low-cost, commercial off-the-shelve (COTS) products for their business functions. These COTS product becomes a system of record (SOR) for the business unit.
Too many companies were saddled with technical and data debt, hampered by existing organizational silos, required to allocate more resources to the core tech stack...
Forrester Predictions 2020. Forrester Research, Inc.
Over time, organizations find themselves using numerous SORs within the enterprise. These SORs store company data in various, often disconnected, locations. Compiling data for business intelligence and other data-driven activities become challenging.
Where is the data? How does it relate? Is the latest version of the data? What story is it telling?
SORs tightly coupled with business functions limit experiences that can be offered to the users. It causes vendor lock-ins since switching SORs without major hindrance to business continuity becomes impossible. Consequently, users adopt means to overcome these limitations by devising “shadow systems” to get their work done. The shadow systems and tools circumvent the system as a workaround for a gap in the functionality. These systems generally consist of spreadsheets, document copies stored on a physical or shared drive, paper copies, email threads, phone calls, etc. Any activity outside of the main enterprise system generally lacks tracking and monitoring leading to more frustration for the users.
A Solution
To make enterprise system performant, scalable, and robust, it can be divided into logical tiers/layers. Engagement, Integration, and Data layers. This post will explore the 3-tier approach that ChaiOne recommends and has implemented for several of our Fortune 500 clients.
With a purpose-built layer structure, the organization can centralize and gain control over the user experience and data orchestration. The data layer consists of all the data sources in the organization, including all the SORs in use. This can include the database servers the SORs use, any REST APIs the SORs may expose, etc.
The integration layer uses API gateways to pull information from multiple sources and presents a holistic picture. These APIs can also pull integrate and pull data from other APIs, cloud sources, etc. Furthermore, APIs can enforce business rules for the applications in the engagement layer.
The engagement layer consists of apps that users can use to pull data from various sources. It allows the enterprise to offer solutions to users to get the true story from their data and simplify decision making. This layer also bridges the gap between business users and IT.
The boundaries between what is IT and what is the business are already fading to reveal a much more powerful and fluid IT capability.
Forrester Predictions 2020. Forrester Research, Inc.
As a side-effect of this separation, swapping out SORs becomes possible and less challenging.
Phased Approach
As enterprise systems mature, data can get fragmented across different sources i.e. SORs, spreadsheets, etc. With each SOR using their own data storage, data entities, relationships, etc., getting a complete picture becomes very challenging, if not impossible. As mentioned in the previous section, introducing an API gateway to build integrations between existing systems – that will not directly integrate otherwise – can help with getting the data from multiple sources, but the transition can be painful for the users.
Change is hard.
Large ships turn slow. The same is true for large organizations. Throughout the organization, plenty of new systems and workflows introduced by business units are met with poor adoption. Often these new systems cause users frustration to increase. As a result, the users abandon them. Any major impact to business workflows demands research, stakeholder buy-ins, value-added scrutiny, and rational emotions to be fully adopted. Therefore, it is beneficial to take a phased approach.
In the initial phases, wean the users off of the shadow systems. Move them to use a centralized solution that bridges any functionality gap. During the transition phases, allow users continued direct access to the SORs to ensure uninterrupted business flow.
Followed by integration with SORs in a gradual manner until the user is fully interacting with the engagement layer only. Users drive the majority of the design and requirements for the engagement layer since they will be using the applications and fill any gap to avoid workarounds. Once the application development cycles mature, the users are comfortable accessing the system only in the engagement layer.
After that, further improvements and enhancements can be applied to these layers.
Single Source of Truth
For organizations with a mature enterprise system and advanced understanding of their data, a holistic data modeling approach such as a Single Source of Truth (SSOT) system can be implemented to improve data quality and accuracy.
An SSOT implementation requires custom data modeling and data relations. Begin by consolidating data models and attributes from all the existing SORs in use into a single data model. Define custom data models to match their business needs. As a result, any solution – internal or external – integrating with the SSOT must be compatible with the APIs and data models. Use data mapping to map SOR entities to custom data models. Finally, position SSOT between the APIs and the data layer. It acts as the data source for the entire enterprise. All data flows into, and from, the SSOT. Most importantly, sync the data – either periodically or by action triggers – between the master database (that holds all the custom data models) and other data sources such as ERP, CRM, etc. to ensure accuracy. SSOT will in covered in detail in another post.
Challenges
As an organization matures its enterprise systems and take control of its data, vendor relations mature as well. When working with vendors, only those vendors that can support integration to the enterprise system are considered. Since not all vendors offer SORs that can integrate externally, vendor options become limited and may have higher licensing costs. However, in the long run, it offers potential savings since the organization is not locked into a single vendor and can alternate to keep costs manageable.
Another challenge is data model management. While reducing dependency on SORs, organizations take charge of their own data modeling and data mapping between SORs. With this power to have control, comes overhead, and data architecture maintenance.
With the right team and resources allocated, an organization can not only tame their enterprise systems to work well, but it can also open up possibilities to make the user experience better. Better user experience, better adoption, better results. Win. Win.
About ChaiOne Corporation
ChaiOne is a pioneer in industrial digitalization. Reimagining plant, field, supply chain processes with design thinking led solutions that reduce cost, increase adoption, and elevate the customer experience. The complete strategy-to-solution expertise sets ChaiOne apart to help industrial customers achieve exponential outcomes across their logistics, supply chain, field, and plant operations. To learn more, please visit www.chaione.com.
