Tuesday, September 22, 2020

Monster Data is Headed Our Way

If you thought Big Data was a challenge for us all, wait until the new wave of Monster Data hits us. We will have to manage it, make decisions, and build processes and applications leveraging this monster data. Just what is monster data, and how will it affect us. 

Monster Data represents data that is overwhelmingly large, unduly complex, can’t be trusted for accuracy. Typically it is composed of multiple kinds of data including structured, unstructured text, voice, image, or video. Some monster data may be unknown or emergent, making it scary to deal with for most individuals, technologies, or organizations.  

Even Larger Volumes        

We have long been concerned about “the IoT Awakening," exposing large amounts of critical data that would likely need immediate attention often at the edge. While managing all the moving parts of Industry 4.0 is a challenge, we see new value chains that employ GPS, tracing, and original digital identities adding data to the mix. As organizations want to leverage data for more refined business outcomes, more data will be needed.

Organizations leverage more powerful AI and computer-analysis techniques to gain insight into human behavior using personality, social, and organizational psychology data. This need will yield data sets that are much larger than what we have today and certainly too large for traditional processes and applications. Data will likely include recorded conversations that could process into usable information.

More and more data is piling up from digital footprints left in social media, cell phones, business transactions in various contexts, shopping, surfing, and other devices that record our every moment, freely given or not. Sometimes this new data is just taken from sites as people pass through, leaving crumbs behind.

Even More Complex  

To order to utilize technology to empower us, the complexity of the data will also become much more diverse. Because large data collections can be computationally analyzed to reveal new signals, patterns, and trends, the complexity of that data will have to be managed well.  Organizations want to deliver insights from human behavior and interactions collected everywhere, every second of the day.

The data will come from various contexts that imply context-sensitive meaning. Hopefully, this new and emergent data will be available on the cloud, but the cost and security issues will make this data more hybrid cloud in nature. The data will likely be a hybrid of structured and unstructured data and require new data management means with ownership and dynamism challenges.

This complex and dynamic set of data sources will become more challenging to manage, but it is on its way to becoming a precious asset that can be leveraged by machine and deep learning. While dynamic and emergent, its use will become more stunning over time.

Even More Inaccurate

Because of speed and size alone, the accuracy of monster data will be a constant challenge. When combining data in new ways understanding its source, context, and ultimate meaning at all levels of granularity, this becomes more of a critical problem for the data management professionals as well as the end-users.

There will be ownership issues and who will be held accountable for the accuracy of any data leveraged. All of this will have to be sorted and managed under the gun with the pressure of speedy results. Of course, internal data sets will have a better-understood pedigree than those data sources from outside an organization and in contexts not well understood.                    

Net; Net:

As we grow to zettabytes, the amount and variety of data being accessed, collected, stored in the cloud, stored on-premises, and analyzed will keep increasing in an exponential fashion. This seems like a near-impossible task until the promise of better analysis and prediction to correct problems takes over our desires. Business outcomes will likely drive this growth in this extreme competition and dynamic environment these days.


Tuesday, September 8, 2020

Decisions Need to Drive Data Science

There has been and will continue to be a significant shift in how data is leveraged in this continually changing world. Until recently, the data science process of collecting, cleaning, exploring, model building, and model deployment ruled the data management mindset. In a world that is "steady as she goes," this makes a great deal of sense. The amount of data to be curated is growing impressively, but the data science mindset is still on the scene dealing though pressed to its limits with big data. Two things will break this sole reliance on the data science process. 

Dynamic Business Scenarios

In a world with operational KPIs staying steady with minor adjustments over time, focusing on data makes sense. That world is virtually gone with the elephant in the room being a pandemics at the moment. Tomorrow it could be natural disasters and the down-stream effects of climate change impacting geopolitical behaviors. There are many business scenario possibilities and combinations headed our way. We can't afford just to explore data and have knee jerk responses.

Monster Data is Lurking

If you think big data is worthy of concern, just think about the monster data just around the corner, driven by higher volumes, more complexity, and even more inaccurate by nature. Organizations are bound and determined to take advantage of behavioral data that is further away from standard core operational data. Monster data includes all kinds of unstructured data that will contain digital footprints worthy of new types of decisions.

Either of these would require a major addition of new data processes, but combined data science processes alone just won't suffice. I am not saying that data science will dim, but it needs some new additional turbocharging and methods that are not just focused on exploring structured and clean data.

Dealing with Changing Scenarios

There are several ways of dealing with scenario planning and practicing responses, but here is what I would encourage organizations to do. Many decisions will drive the data that is leveraged during these efforts.

  • ·        Plan probable scenarios by having executives brainstorm and list likely scenarios and their outcomes.
  • ·        Simulate and practice these likely scenarios, so they become part of the muscle memory of an organization. It will involve leveraging key data sources cascading to tactics and operations. Build communications mechanisms ahead of time and communicate readiness.
  • ·        Identify unlikely dangerous scenarios and simulate the effects and plan responses appropriately.
  • ·        Identify critical decisions, events, and patterns to scour appropriate data resources (owned or not).
  • ·        Identify key leverage points in processes, systems, applications, and the data that could be involved

Dealing with Changing Tactics

Middle management is always trying to optimize outcomes for their functional areas though savvy organizations try to link results to remove friction points for overall optimization. Optimization often leads to self-imposed changing goals that need to be operationalized or tweaked in operations. When executives want different outcomes based on a refined organizational charter, new governance rules, and critical trends delivered by business scenarios in place, a bigger picture is in play. Tactics are the essential glue to hold together operational outcomes guided by goals. As these goals shift in a dynamic set of business demands, managers would be wise to be ready for new guidance coming at faster speeds by following this list of practices.

  • ·        Understand the impact of significant changes by modeling or simulating the effects of change.
  • ·        Be aware of all executive expected sets of scenarios and search for critical events and patterns to detect new scenario emergence.
  • ·        Implement various approaches to near real-time responses, including digital war rooms, dynamic process/application changes, and low-code methods.

Dealing with Operational Change:

Typically operational processes and systems are in place to deal with day to day operations. Changes in behaviors, markets, tactics, or scenarios will cascade down to operations. There may be additions and changes to procedures dictated by outside factors over the routine operational optimizations that occur on an ongoing basis. Processes tend to be more stable, but some changes could rock the house. To deal with functional change, I would encourage the following activities.

  • ·        Model key decisions that affect KPIs and desired business outcomes.
  • ·        Generate procedures from the models—manuals for human resources and code for processes and applications.
  • ·        Perform a volatility analysis based on past changes to identify hot spots. Enable hot spots for change, particularly for code using late binding techniques or low-code.

Net; Net:

We are entering an era of significant change linked to constant change. It means that just shining and studying data alone does cut it as a sole strategy. While data affects decisions as they are made, deciding what is going to change is emerging as a dominant new organizational competency area. We need to add some new disciples/practices to thrive going forward and call it Decision Science. 





Wednesday, August 26, 2020

Budgeting Technologies for 2021

 Organizations are now challenged in new ways; therefore, they must budget very carefully, as we advance. There will be a tug of war between accelerating digital and dealing with budget reductions for IT investment. Gartner, affectionately known as "The Big G" in my circles, has predicted IT budget reductions. At the same time, businesses are being pushed to accelerate digital transformation. Savvy organizations will save with technology to invest in technology to break through this set of apparent conflicting goals. Organizations will be careful in deciding what to invest in to survive, thrive, and capitalize on these dynamic and challenging times. I will try to lay out the five most important technologies to invest in to keep these conflicting goals in balance.

Continuous Intelligent Automation & Cost Optimization

Automation has come on strong through the use of RPA, Workflow/iBPMS, and Low Code solutions as of late. Now I see an extension of the accelerated use of both guided by both Process Mining and AI. Continuously, process mining offers extreme visibility for opportunities to handle outliers or optimize on process/case yielding time and labor savings. Machine and Deep Learning will also play a guiding role in finding more optimization opportunities over and above what the human eye can detect across various mining visualizations. The pressure for quick improvements with fast feedback cycles will push more detection of options to intelligent software or machines as responsible AI continues to develop. The savings from these profitable efforts can be applied to future digitization efforts. These efforts can be multiplied by using Software as a Service (SaaS) in some instances.

Human Augmentation & Skills Expansion

As more automation pushes humans to higher-skilled pattern detection, advanced Decision Intelligence, and smarter actions, humans will use technology to enhance a person's cognitive and physical experiences. There may be sensory augmentation, perception augmentation, and AI cognitive assists in enabling higher-skilled work levels. In the case of physical responses, appendage assistance, and exoskeleton leverage may be enabled. Imagine having the assistance of experienced experts in your ear, eye, or mind to accomplish more challenging tasks. The new worker will have interactions with technologies that will enable super skills and accelerated outcomes. This will start small and accelerated by the end of 2021.

Immersive Experiences & Visibility

All constituents will have more immersive and pleasing experiences, making them more informed and satisfied. Customer Journey Mapping/Mining technologies will allow organizations to get real-time and truthful feedback from their customers, employees, partners, and vendors to help improve their experiences on a continuous basis. Virtual Reality and Mixed Reality has the potential to radically influence the direction of improved customer experiences, product supply, and value chain services. These new visibility assists can give customers a real sense of progress towards their outcomes when balanced with organizational processes.  Some organizations have seen the value in onboarding and immersive training in a safe and realistic virtual environment.

Augmented & Real-Time Data Management

The amount and speed of data are increasing faster than our ability to manage it. Big data is turning to a complex and multi-head monster of data types with various different requirements. Managing all the data and data types will require assistance. Data marketplaces and Exchanges are emerging to add to the data chaos. Managing the various data sources will need to leverage the Database of Now integrating various data sources in the cloud, AI's ability to learn from the incoming flood of data, and the metadata that defines it within its various contexts and workloads are essential. Dark data will start to be better understood. Data journeys and transparency will be assisted by practical Blockchain that enables data traceability.

Autonomous Bots & Edge Computing

Autonomous bots/agents that bid on work at the maximum and minimally perform activities on "the edge" with AI help. Edge processing, data collections, and decisions are placed closer to the information and activity source to sense, decide, and respond closer and in the proper context. Often the IoT is where this occurs when machines, sensors, and controllers are involved with physical activity, but there are instances software, Digital Twin, or not have a presence at the edge. Often these activities are semi-autonomous and supervised today, but we are moving to more autonomy over time. Smart spaces, smart production, and smart value chains will drive these kinds of efforts. Look for robots as a service (RaaS) to elevate some of the data density issues.

Net; Net:

Every organization will have to match their operating plans to the technologies above and decide what they want to take on within their cultural and risk limits. The danger here is to focus on technologies that can contribute to short term financial results to the detriment of the future. This is true with short term cloud efforts without thinking of the total cost of ownership. Grab some profits, but invest wisely to compete digitally in the future. Negotiate with your financial folks, please or hope they get more innovative.








Thursday, August 20, 2020

Are Masks the New Accessory?

When COVID 19 first emerged, a number of us scrambled to get any mask we could. In our home, we first went for the standard paper mask from the Pharmacy. As COVID 19 got to be a bit more pervasive and scary, we upgraded to N95s or K95s and some double layered cloth masks, mostly in black. Now masks are getting better looking, so I thought I'd put some of my art on masks to see if there was a demand. Indeed there was. It seems the folks that want masks, want something nice to look at. Here are the masks that I'm offering. I can be reached through if you are so to give me feedback or even have one in your possession. You can see more of my art by clicking here

Wednesday, August 19, 2020

Acceleration of Decisions Helped by the Database of Now

Things were going along nicely until COVID 19 hit, and it was “game on” for rapid decision making. Executives were slammed from operational optimization with known decision/static models while transforming incrementally to digital towards a world of large amounts of decisions made in short time frames. The vast majority of organizations had not planned for this kind of scenario, thus not practiced to handling it. The acceleration to some form of digital and remote workers was instant. Our executives were bombarded with one critical decision opportunity after another, and many were up to the task thankfully. Is the question "Is this a one-off situation"? I would argue that maybe not the exact same scenario, but the beginning of many emergent situations at various corporate performance levels. How does the Database of Now help

Integrated Data to Support the Lateral Thinking in Decision Making

Traditionally decisions have been generated by new management goals and finding "aha" discoveries in data? While this approach will continue, forced innovation will be a necessity driven by these large-scale and emergent scenarios, such as changing markets, customer demands, extreme competition, and the desire for better outcomes. This shift will require more data and new complexities to continue to monitor, decide, and take appropriate actions. Intercepting the changing future will also drive towards adapting to and integrating new data resources, many of which will be cloud resident. The Database of Now supports dynamic and easy integration of new/emergent data sources.

Fast Data for Analytic Assistance, Guided, and Flexible Implementations

Decision-makers will demand assistance in making informed decisions fast and understanding the ultimate impact of their actions in planning and execution modes. The first demand will be for fast data supportive before the actual decision occurs. Lack of speed kills but so does speed without anticipating potential outcomes. Analytics will greatly assist decision-makers in understanding the possible consequences of their impending decisions. Once the decisions are made, the emergent effect will need to be tracked, monitored, and measured during rollout. Once implemented, fast feedback loops will help guide adjustments for better performance while sensing emergent patterns for potential new decisions. The Database of Now is designed for speed.

Smart Data that Leverages Machine Learning and Other Forms of AI

Today, most decision-makers are highly involved with the decision-making process unless they can be easily automated, usually using decision models. Typically these decisions are operational and static in nature, but there is a strong trend towards flexible change and emergent business outcomes are driven by new responses to integrated response scenarios. Either way, forms of AI can speed the decision-making process, starting with machine learning that watches conditions and outcomes. Deep learning can sharpen the focus for even better results. This kind of leverage is often described as smart data commonly used in supervised learning situations. With the advent of emergent and complex conditions driven by expected or unexpected events and emerging patterns, AI will take a more judgmental role in an unsupervised fashion. The Database of Now is ideal for having the most up to data and contextually sensitive data sources necessary for high intelligence.

Net: Net:

The future will require rapid decision making that needs speedy data that traverses many data types, monster data volumes, and growing complexity.  There will be quiet periods of optimization that will also benefit from the Database of Now, but get ready for waves of emergent situations potentially never seen before by the modern decision-makers. It may turn decision making on its head changing from only modeling operational decisions into crisp responses also to include emergent decisions dependent on complex, fast, and shifting data sources. Will you be ready for fast and effective decisions for customer needs and operational effectiveness?

  •            Customer experience demands responsive and instantaneous data.
  •            Business operations insights enable instant adaptations for changing market needs.

Additional Reading:

IncreasingCorporate Performance with the Database of Now  

Context: TheConnecting Clues for Data  

DeliveringSuccess with Smart Data Streams  



Monday, August 3, 2020

Increasing Corporate Performance with the Database of Now

Organizations no longer have the luxury of sitting back and waiting for an opportunity to react. Corporate performance depends on intercepting the emerging future quickly, thus putting a premium on the Database of Now. We can see many examples of the inability to pre-build strategic responses to emerging conditions such as inverted yield curves, new super competitors, hyper disinflation, currency shifts, pandemics, ECO events, and geopolitical shifts. So how do organizations take advantage of the database of now and build for interacting response cycles? The answer is to create a database of now and leverage it differently at different levels in the organization (See Figure 1) while trying to extend reaction to preemption. The interaction will be changed at different levels and cascading levels of strategy, tactics, and operations.

Figure 1 Interacting Response Cycles

Organizations are running in an automatic mode within normal conditions; they take actions without a lot of thinking or bother. The problem today is that automatic mode is not happening consistently with profitability like it has in the past because of emergent conditions. These conditions can emerge from a variety of sources represented by fast and large growing sources of data. These conditions can come from outside the organization in either an anticipated or unanticipated manner where they are not as controllable. These conditions can come from inside the organization to optimize business outcomes through observation or management influence in a controlled fashion. See figure 2 for the common sources of new conditions. The causes include changes in data, patterns, contexts, decision parameters, results from actions, changes in goals, or new risk management desires/demands. These changes are occurring on a more frequent basis and at a faster speed, thus creating the need for the Database of Now

Figure 2. Sources of Emergent Conditions

Keep in mind that each level's triggers in Figure 1 will likely be different, iterative, and possibly influenced/interconnected by other levels.

Operations of Now:

Operations are focused on completing business events, customer journeys, and work journeys with the support of humans, software, bots, and physical infrastructure. The operations are often iterative and monitored in a near real-time fashion. Today's operations require a Database of Now where the dashboards reflect actual progress/completion of work. When exceptions emerge, responses are required within the constraints of existing operational goals to make minor adjustments. Also, significant adjustments need projects that may leverage a fail-fast approach to make corrections. Operational goals are often influenced by changes initiated by tactical and strategic decisions and adjustments. Analysis and reporting help make appropriate adjustments without unseating other operations.

Tactics of Now:

Tactical management within the constraints of strategy tends to optimize interrelated outcomes that may look across multiple operational domains. Real-time forecasting based on real-time data is essential to predict the direction of aggregated operations. The Database of Now plays a crucial role in making better decisions by quickly changing rules to optimize business outcomes in support of strategic goals and directions. Tactical changes may imply shifting resources, changing rules, goals, and constraints of aggregated operations. Often key projects identified at this level, like recognizing patterns that might indicate the need for a new product or service. This is also the level that decides the amount and type of automation that will help reach the currently selected strategy.

The Strategy of Now:

Strategies tend to stay stable and are highly linked to the organization's missional operations within Its typical communities and common scenarios. Predictive and prescriptive analytics help shape expected scenarios that may be sitting on the shelf with their associated tactics and operations ready to jump in at a moment's notice. Of course, the Database of Now can point to a playbook for switching scenarios when expected patterns emerge. Still, unexpected patterns can generate the need to apply new scenarios generated by more predictive and prescriptive analytics. 

Net; Net:

It is pretty easy to see that fast monster data will create the need for the Database of Now necessary for better performance at all levels (strategy, tactics, and operations). It is also clear that fast data without time lags generated by too many synchronizations and transformations is necessary for better corporate performance while keeping all contributing resources aimed at business outcomes. 



Tuesday, July 14, 2020

Best Visual Options for Process Mining

Until bots can be cognitive enough to complete closed-loop improvements on processes or data stores on their own, visualization for humans will be key for making process improvements. Today many of those improvements are made through data mining in real-time or after the fact by humans. They do it by setting tolerances and monitoring outcomes or looking at the visualization of process instances that travel through processes or collaborations. The best visual options for any organization will depend on their culture, maturity, and desired business outcomes. I've laid out three categories of process mining visualization techniques that typically match maturity levels. I have used examples from vendors to help sort out the options, so your favorite vendor may have been left out of this post. 

Basic Visualizations

Basic visual analysis sometimes starts with an ideal process, sometimes called a "happy path", and look for the actual paths taken by a process. Organizations sometimes start with the outliers and try to reign them in closer to the ideal. Other organizations start with clusters of most common deviant paths and try to improve them. See the visualization below for a representation of this approach. Most organizations do a before and after to measure change effects, also depicted below. This shows the process before changes are made and the resulting process with deltas in certain instances. 

Intermediate Vizualizations

More mature organizations try to add important business contexts to show the actual delivery made by processes in terms of key measures. One of the more important contexts, shown below, are the actions shown on a timeline. This gives "time to results" a high priority while counting key costs and resource utilization specifics. This is an effective way to eye-ball opportunities. Another key approach is to show the process instances in light of desired outcomes versus real outcomes usually represented by dashboards or scorecards also depicted below. This is the start of the journey to adding more intelligence to the process of mining efforts. Simple Step through visualization with or without simulation of proposed changes is another nifty approach pictured below. 


Advanced Visualizations

One of the proven visualization techniques is animations that attract humans to opportunities through either speed or color indicators. This typically shows choke points and bottlenecks, but there are additional uses to simulate alternatives to show the value of different change opportunities. See below for an example. Predictive analytics combined with virtual reality can be used to visualize points of view or personas to fine-tune processes from different perspectives walking through a process or journey as depicted below. For those organizations that want to learn as they go, they can add machine or deep learning to improve processes as depicted below. 

Net; Net:

The visualization approaches can have a great impact on the resulting processes and finding opportunities for more automation, tuning for better results, and trying alternatives without the negative impacts of breaking or breaking optimized processes. Your chosen visualization might be a personal preference, but as organizations mature more sophisticated visualizations will be needed until the smart autonomous bots or agents can do this work as a partner or autonomously.