What’s Driving Data-Intensive Applications?

Today and in the foreseeable future, huge waves of data-intensive applications are breaking over us, with more waves to come.

It’s not just the data volume, often referred to as "Big Data" or "Monster Data," which pushes opportunities in the direction of organizations. It’s also the demand-pull of applications, processes, and journeys growing in importance for organizations to compete. These data sources are often measured in terabytes or petabytes, but “being large” is just the obvious, in-your-face description of what comprises a data-intensive application. In these apps, the data is commonly persisted in several formats and distributed in many locations and must be cared for in various ways for organizations to flourish. Coping mechanisms will be described in future posts, but this post identifies the drivers of data-intensive applications.

The Demand-Pull Drivers are Data Hungry

Because of the pressures on organizations to expand their views on the scope and impact of applications, there is a considerable demand for more data as focused and simplistic applications transition to intelligent large-span applications. In addition, the speed to detect emergent signals, events, and patterns is ever-increasing, putting pressure on follow-up decisions and appropriate actions. It’s much like a fighter plane that has to make decisions and take steps in seconds; however, management is used to working in days, weeks, or months.

Moving from Dashboards to Fast Boards

Today's organizations need to anticipate critical patterns to intercept opportunities and threats at more incredible speeds to make decisions and take appropriate actions. Some organizations are crafting technical sentinels that sit on the edge to sense and sometimes respond if given the freedom to do so.

Excellence with Management Cockpits

The idea of people watching many individual dashboards/fast boards and integrating their contexts with speed is somewhat an unrealistic expectation. Minimally, these need to be brought together into a management cockpit to grok the intersections of the visible measures. These measures range from KPIs to out-of-tolerance situations. Eventually, the management cockpit would be assisted by bots/agents to notify management of threats and opportunities. Ideally, these management cockpits could help in a "fly-by-wire" fashion within practiced business scenarios. 

Decision Management and Assistance

Besides the speed to sense, decide and respond, data has to be available to venture into new contexts to aid the decision-making process and play out the ramifications of any action about to be taken. Operational adjustments may require simple tuning or kick-off other individual efforts. Tactical moves require new versions of rules and critical adjustments of guardrails and constraints resulting from decisions. Strategic moves require some forms of advanced analytics and potentially gaming alternatives through a management cockpit. 

Value Chain & Supply Chain Extensions

Today, an awakening occurs that requires knocking down organizational/skill walls to eliminate silo thinking and actions. There is a race to kill silos in value-chain and supply-chain situations encouraged by businesses partnering to produce products or services. There is a premium on innovative collaboration that crosses all kinds of boundaries to create overarching goals and results while satisfying individual organization units at the same time. The goals, rules, policies, and constraints need to be tweaked simultaneously during operations in the middle of changing conditions. 

Supporting  Journeys

There is a considerable push to define constituent journeys, especially customer journeys, that are often integrated with employee/support journeys. Journeys require an outside-in perspective requiring more data to represent specific goals of the personas and individuals interacting with an organization. The customer experience is tracked, measured, and recorded with sentiment data, often represented by voice interactions when live via a representative or chatbots. The data around loyalty and satisfaction proliferate with an outside-in journey perspective.

 

The Data Push Drivers Overflow

Data offers opportunities in its new forms, amounts, locations, and captured contexts. Until now, the "Big Data'' headline has been driven mostly by the volume story. That is about to change with the new data types and formats that are entering the organization. In addition, there is a new generation of the distributed types and a movement that says that views can be constructed no matter where the data resides at the moment. While location complicates data quality and compatibility issues, there are alternative ways to cope with new tolerances for perfection depending on the usage described in the demand-pull section above.

 

Voice Data

Voice is a key new tributary to tap for organizations, thus tempting leverage for competitive advantage, particularly in servicing processes/applications. Voice can be leveraged to see how often competitors are mentioned in calls. Voice can also be analyzed for emotional reactions in the context of the servicing experience. It is often helpful in unscripted situations, which often occur with service representatives who are skill specialized. Now it’s not just NPS scores that count for customer satisfaction measurements.

Image Data

Images can be helpful when brought up in context. Not only can physical plant layouts and machinery be checked for safety purposes through image analysis, but broken machinery can also be detected before a significant or cascading problem occurs. Not only can out-of-bounds situations be seen, but optimal real-time planning can be enhanced by image detection. Real-time image help is a must for some jobs.

Video Data

Videos can be leveraged for better productivity, such as optimizing worker movement for better quality and faster processing. Video can be used to identify resources in action, such as people and machinery, for various kinds of operational optimization and training opportunities. Imagine showing an inexperienced worker a video on how to service a particular component such as a pump after taking a video of it in operation provides self-paced, on-demand training - without asking experienced workers for help.

Edge Computing Data

Data can be detected at the edge before it hits mainstream processes/applications. An unexpected event can trigger the notification of an emergent set of conditions or patterns for real-time decisions or actions in most cases. With the proper freedom levels, goals, and guardrails, this reporting creates a bevy of data for each node at the edge, whether a sentinel or an actor.

Distributed Meta-data

The data about the data is often called meta-data. With the sophistication of data storage and state combinations, meta-data is important to properly manage the distributed data sources. In addition to the physical state of data, such as its location (on-premise or cloud storage), its state of meaning, transformation, source, and context can be managed with meta-data.

Net; Net:

There are so many trends contributing to data-intensive applications that there will likely be another new one later today. New ways to manage traditional and new coping mechanisms will emerge over the next few years. I'm betting that both the demand-pull and the data push drivers will only accelerate. We are in the middle of a massive revolution for managing data differently. Be aware and get ready - data-intensive applications are coming for your organization.