AI: For Good or Evil?

AI has significant momentum now and contributes positively to businesses and everyday life. Today, industry and government leaders warn of the dangers of unbridled AI. So, let's peel back the onion on AI for Good and AI for Evil. Next, let's see what it takes to steer AI positively with as few side effects as possible. We all know all advancements come with good and bad effects. Look at the automobile, for instance. Autos take us to many places, but driving them unsafely without following the rules of the road leads to injury and even death.

AI Brings Good for Many Industries.

· Healthcare uses AI for preventive medicine, advanced diagnosis, personalized treatment plans, and drug discovery.

· Education uses AI for lifelong learning, adapting to changing career changes, shifting to new opportunities, and personal interests with virtual tutors and dynamic personalization.

· Transportation uses AI to optimize the planning and operation of smart cities, support various levels of autonomous vehicles, and optimize eco outcomes within the need for goal-directed efficiencies.

· Finance uses AI for investment management, wealth management, and fraud detection.

· Customer Service uses AI for hyper-personalization, sentiment analysis, and virtual assistants.

· Agriculture uses AI for sustainable farming by optimizing resource uses, automated or not, reducing waste, developing crops for climate change, and practicing sustainable soil management.

· Environmental Protection uses AI to design and implement effective climate change mitigation strategies, biodiversity monitoring, and resource management.

· Manufacturing uses AI in smart factories for optimization, efficient supply chains, and innovative material discovery.

· Entertainment uses AI for immersive experiences, innovative content creation, and audience engagement.

· Accessibility uses AI for inclusive design, enhanced communication across language barriers, and assistive technologies for the less capable.


AI Brings Good for Businesses

Businesses, in general, are using AI for enhanced decision-making in both a proactive and reactive manner. They are Improving the customer experience with AI while increasing their operational efficiency in a balanced fashion with AI. Marketing and sales are expanding their reach through better targeting and predictive forecasting. Businesses are creating new products and services with AI while better supporting their existing portfolio of products and services. AI helps with human resources with better engagement and automated recruitment. AI helps with fraud detection, compliance, and speedier governance. Businesses leverage AI for better expense management and investment strategies. Organizations can predict customer churn and measure customer sentiment in real time. Speaking of real-time, threat detection and vulnerability management can take advantage of AI. Businesses can increase their productivity, revenue, and costs while leading in sustainability through resource optimization and sustainability. Companies can take great advantage of various types of AI.

AI Brings Good for the Consumer

Consumers are experiencing many benefits of AI today, and AI is leading to even more benefits. Personalization provides tailored recommendations, customized voice/language engagement, and satisfaction with 24/7 availability and quick responses to overall needs, not just transactions. The shopping experience is improving with virtual try-ons and augmented/enhanced reality. Health assistants with links to telemedicine will help with preliminary notice of symptoms and diagnosis. Financial advice will be provided to optimize customer goals, both long and short-term. Recommendations for targets in the tsunami of content emerging to help the viewing/listening experiences. Home management and security will benefit from AI as well. Consumers will benefit from sustainability suggestions as well. Civic engagement, cultural preservation, and mental well-being are also benefits.

It's hard to argue that AI is not used for good and that the expectations for more AI benefits are sky-high. Yet, at the same time, some horror is dribbling from under AI. There are bad actors out there trying to do evil with AI. Without all the rules of the AI road laid out yet, there is an opportunity for these bad actors. Some individuals use AI to gain advantage, leverage, and illegal financial gain. Some use AI to subvert power, weaponizing AI for offensive purposes.

AI Enables Bad Actors

· An early evil use of AI is for cyberattacks that target critical infrastructure, financial systems, or government systems for monetary gain, espionage, or sabotage.

· AI can bring social engineering and manipulation by using social engineering techniques to manipulate individuals, influence public opinions, and spread misinformation for financial or ideological gains.

· AI can perform mass surveillance, tracking individual movements, communications, and activities without their knowledge. AI can infringe on privacy rights and civil liberties.

· AI can power autonomous weapons and other forms of lethal weapon systems without human intervention. Drones or robotic soldiers are examples of weapons that could act alone or swarm to escalate conflicts or undermine international stability and security.

· AI can be used by adversaries, including cybercriminals, state actors, and terrorist organizations, to create a movement against established society.

· AI can inadvertently exhibit unintended behaviors or consequences that manifest as errors, biases, or impacts on individuals, organizations, or society at large.

· AI can be harnessed to perform financial crimes and fraud toward individuals or organizations.

· AI can be used to create deepfakes and misinformation to gain financial advantage or take down the reputations of individuals or organizations.

· AI can affect biases and exacerbate inequalities by targeting individuals or groups to displace people from jobs or make it hard to thrive.

· In the worst-case scenario, AI could pose existential risks to humanity if misused or developed with inadequate safeguards.

AI Enables Power Plays

Geopolitical competition will drive rivalries among nations, pushing them to vie for dominance and try to leverage AI for innovation for economic gain and technological and military advantages. While not all this is bad, it can be taken to extremes, leading to new global pressure points and chess matches. Some of it could lead to arms proliferation and a new arms race. The amount of cyberwarfare and spying is bound to increase. The real risk is the need for clear regulations, guidelines, and treaties. There is likely to be a blurring of AI for military and civilian uses that will also muddy the mix. There is also a real danger of a supervillain or group leveraging AI to extort or unduly influence others.

Net; Net:

The fear of Evil AI will not drown out Good AI for now. The control of autonomous weapons, cyber warfare, aggressive intelligence, and psychological operations should be planned for the future. Establishing international regulations, norms, and treaties is the best way forward. Agreeing on what ethical AI development is and monitoring it with transparency. A move towards more “human-in-the-loop” for lethal actions is needed. Establishing robust oversight and governance and promoting peace and diplomacy with AI can work for us. The proactive measures of establishing robust international regulations, enforcing ethical guidelines, promoting transparency, and fostering a culture of peace and diplomacy can mitigate risks and bad behavior from bad actors. The alternative is mutually assured destruction, as we have with nuclear powers.

Additional Reading:

Definition of AI

Future of AI

Fear of AI 

AI Inside Out  

AI Tributaries

Let’s Get Data Tastic in 2022

It would be easy to focus on the data challenges facing organizations today and respond reactively to them. However, we all see the problem of massive amounts of data coming in faster than we can deal with it. We are all learning how to cope, but I think 2022 is the year organizations make some headway on getting ahead of this problem and start making new opportunities for themselves. I want to enumerate some practical things we can do in this post. One is to work towards building a better data foundation by augmenting and modernizing towards an authentic data fabric/mesh, and another is enabling automated learning to find the data nuggets that are candidates to leverage and finally looking at new ways to gain business advantage by leveraging data.

Build, Augment and Modernize

All organizations can make their data resources better. The opportunity here is immediate and multi-faceted as organizations build toward data fabrics and meshes. The core of a proper data fabric would be a unified database that can handle multiple different data styles of operations, including transactional and analytical with the same data. It will help reduce the number of copies of data necessary for different kinds of operations. A database that can work efficiently and seamlessly with hybrid and multi-cloud situations is a minimum price of admission these days. A database that can mix real-time data with archival data is a must. It requires organizations to move from specialty databases on-prem to a real-time catalog-driven approach that finds and data with different speeds, formats, and data types and leverages that data. All of this is for better decisions and better handling of emergent situations while dealing with limitless speeds/feeds. This technology exists and is growing more capable of supporting an actual data fabric/mesh needs to cope with businesses.

Automated Learning

With the recent strong growth of AI and Data Mining, organizations are tasting some of the early benefits of learning from data leveraging emerging digital technologies. Getting a handle on this automated learning will be table stakes for survival going forward. Even armed with algorithms, people can't keep up with the influx, speed, and variety of new data. Trained machine learning algorithms can understand many data channels such as email, chat, speech, and image sources. This kind of AI, combined with computational statistics, can help find opportunities or threats in the future. Data mining can explore data leveraging unsupervised learning to visualize and provide patterns of interest. Advanced data sifting can employ neural networks to find opportunities in the tidal wave of data.

Gaining Business Advantage

I think organizations have done a great job of using data to gather leads and close sales to increase the odds of growing revenue. Still, there are more opportunities beyond micromanaging resources. While optimizing resources is still a top priority, organizations are encouraged to expand their view to include opportunities for automation, better interact with customers, use digital twin opportunities for better visibility, and leverage data to better manage emerging situations detected in existing or new data sources (voice, image, video, GPS, etc.). It is the more difficult area to move forward in as a leader because there will be some pioneering and risk unless your competition does it first to show the way.

Net; Net:

Organizations must have activity on all three of the above paths to better data utilization; however, the priorities will vary by organizational culture (risk-averse or not). For example, assertive organizations that want to capitalize on data will gain business advantage first. Still, most organizations will grow into an advantage from the bottom up by learning to leverage a new generation of databases inside a morphing data fabric/ mesh. In parallel, most organizations will be growing their capabilities in automated learning opportunities starting with low-hanging fruit.



Additional Reading:

Data-Intensive Applications

Reducing Data Sprawl

Data Infrastructure Debt

Unified Databases

Real-Time Data

The Rise of the Unified Database

We are living in the days of ever-expanding data sprawl caused by many hard to control factors. Data sizes are monstrous, the speed of data ingestion is ever increasing, the demand for better latencies is insatiable, and the complexity of the data is increasing. All of this while the need for custom views accessing the same information is exploding.

There are almost too many options that require specialized technical skills to create complex integrations that produce performance bottlenecks when working. It becomes even more nightmarish when testing and troubleshooting. Is the problem hopeless for organizations trying to manage and utilize their data? The answer lies in unified data and unified databases. Let's dig a bit into unified databases and how they relate to the holy grail of an effective data mesh.

What is Unified Data/Database?

A unified data model is a crucial piece to simpler data architecture and supports the data mesh. A unified data model offers an opportunity for organizations to analyze and operate on data from multiple sources in the context of shared views supporting shared business initiatives. The unified data model bridges different ecosystems, allowing organizations to contextualize data sources across various services. The task is statically or dynamically mapping each dataset to a more singular schema or view.

The Benefits of Unified Data/Databases

The benefit of data unification is that it provides a more holistic and accurate view of your many data sources while operating on them in different modes with different performance characteristics. The unified database can simplify the multiple tool and database environment and create a common denominator for data use without worrying about the scale. Unified databases can make data more practical and actionable while helping the data accuracy with the least energy by users and the data architecture keepers.

Downsides of Todays Data Architectures

Today organizations most often run-on specialty databases that have to be synchronized periodically. With the demand for more real-time management, this becomes a copy carousel that gets in the way. It leads to high-latency data makes always-on systems a nightmare. Because different data sources on different data architectures will require redundant transformation logic during the copying and transformation processes, often it isn't easy to guarantee consistency. It also becomes a challenge for data security and privacy programs. What's worse is that some of these copy/transformation processes fail, creating cascading delays and errors. Because our databases do specialized work enumerated below, the copying process is a never-ending nightmare. A unified database that can nearly do all the work of these specialized databases makes for a simpler world.

Generalized Databases

Typically generalized databases are for analytical purposes and rarely scale for transactional purposes. They are often easy to use and handle lower scales of data. Unfortunately, the complex uses and monster data tend to bog these analytically focused generalized databases.

Transactional Databases

Typically, transactional databases are focused on high-end operations where volume and efficiency are often desired. They can stand up to volumes generated by nano-second demands and work well in long logical work units for tasks and processes. However, they aren't for end users who often desire analytical feats leveraging siloed operational data where lateral views are not easily supported.

Analytical Databases

These databases are perfect for high-end analytical work where many sources are mainly focused on reading data but not necessarily creation, updates, or cascading deletes. However, they aren't as easy to use as generalized databases, so they are their own breed

Net; Net:

The unified database does a great job of spanning all these specialized databases. It often ends up as a hub through which many divergent views are connected to many divergent data sources of various types. It simplifies work and gets organizations off the copy carousel. The Unified database is an obvious choice as an infrastructural centerpiece for an organization's data mesh. It makes the shift from inflexible centralized data infrastructures to a data mesh that supports distributed “data-as-a-product” This is particularly important with a distributed data mesh that spans the clouds and on-prem.



Additional Reading:

Get Ready for the Big Shift to the Data Mesh

Convergent Data is Here to Stay

Real-Time Data Mixes Well with Archival Data Now

Leveraging Hybrid Cloud & Multi-Cloud

Speed, Scale & Agility Delivered with Distributed Joins

What’s Driving Data-Intensive Applications?

Reducing Data Sprawl

What are People Reading in 2021?

 We have three-quarters completed in 2021 and people are shifting their topics of interest. Here is a visualization of the activity on my blog. The trends are shifting based on readers' interests. Jumping to the number one issue is Results Oriented Communications in the age of Hybrid Work. While Customer Journeys are still of interest, they slipped to the number two issue. The next highest topic of interest is Management Visibility represented by the Management Cockpit and Real-Time Fast Boards. The next trend revolves around just what is Digital Transformation and where to put Digital Investments. The last in the top five trends revolve around Data Sprawl and Data Fabrics.  If you think there are other high-priority topics, you can put a comment on this post or hit me up on Linkedin. Below is a graphic depicting the activity for the last 12 months.

There is an interesting trend in offshore interests. Northern and Eastern Europe activity is on a marked increase for the year. The Nordic countries tend to be on the leading edge along with Germany. See the activity by a specific country other than the US & Russia below.

Looking at the last 6 months the digital investment topics seem to be on the rise.  

Net; Net:

I'm thankful for my loyal readership and would love any feedback you have on my analysis or ideas for new blogs. The activity is headed towards 800K in 2022 cumulative since mid-2013.  

Data Infrastructure Debt is Hampering Business Returns

There is a great deal of effort and cost associated with keeping technological components up to date and easy to use for more organizational leverage. Some folks estimate that this can reach near 90% of the budget allocated for technology. Every time we add another technological component, the more debt an organization builds. It is excruciating for data infrastructures as data can be used repeatedly to leverage business benefits. It gets in the way of delivering the newly desired outcomes of better customer journeys, more automation for cost savings, and leveraging an organization's resources in new and innovative ways. This post will concentrate on the data infrastructure portion of the technical dept. At the same time, it explores the causes, adverse effects, and how to solve them for data infrastructures and the ever-growing data sprawl.

Data Infrastructure Debt Causes

•        An industry trend implies that modern applications need to be built on top of one or more special-purpose databases, thus adding more technical debt for each database. What makes this particularly difficult when combining and translating data for more use and leverage.
•        Over the past decade, applications have become more data-hungry themselves. As a result, they require particular data dynamics, analytics, and models, implying more cross-use of existing data and new and unique databases.
•         The explosion of free software tempts folks to leverage what software is already there, leading to more unique data infrastructure components. As a result, it creates a specialty database explosion.

Effects of Data Infrastructure Debt

•         Data infrastructure debt is insidious and accumulates fast, and is very difficult to undo. It is a multiplier to the existing debt dragging back initiatives that organizations want.
•         Initiatives that are given the green light are slowed down by the need to aggregate and translate data from many more sources than it would typically take. 
•        Data infrastructure debt keeps the CIO on the sidelines instead of in the critical driver seat for digital efforts in the future. When the CIO is fighting a cost war instead of a results war, digital efforts wane in priority. 
•         We have to hire all the specialty skills to keep the variety of special purpose and legacy databases.
•         The drag causes businesses to set up their own technology efforts. But, unfortunately, they are often naïve to the problems they are creating and throw the debt over the fence to the CIO after making decisions the CIO would not have allowed.

How to Start Solving Data Debt?       

•         It is essential to stop on-boarding new databases even though the software that comes with the new data might be free or low-cost. 
•         Assuming organizations do not want to rip and replace databases immediately but slowly retire many, buying a modern DBMS that works well in the cloud as a No-SQL approach while still supporting SQL commands and databases is ideal.
•         It is not by giving up on innovations and digital transformation efforts by prioritizing these great business outcome efforts overpaying past technical debt. However, it may mean not taking advantage of new features on old databases.

Net; Net:

Smart CIOs are reaching for single modern, scalable relational databases that can support the many needs of applications. It would include mixing real-time data with operational, warehouse, big, and archive data with ease. These databases can operate across cloud providers and on-premises. There is a big trend towards capable databases that can act as a data mesh while reducing the data infrastructure technical debt that promotes unnecessary data sprawl and the difficulties with protecting and leveraging that same data.

Reducing Data Sprawl

 Data sprawl is everywhere and is becoming a bigger problem by the second as we move into a near real-time world. It hits people and organizations, but organizations are on the leading edge to respond to it. Data sprawl refers to the ever-growing amount of data produced, dealt with or aggregated from various new contexts, events, and patterns. It is often mentioned as "big data," but I prefer calling it monster data because of its sizable increase and speed of propagation. It is usually spread over multiple data storage types, networks, and applications that grow as new technologies and data types are introduced. This short blog will cover the significant sources of data sprawl, the considerable effects of the sprawl, and the meaningful ways of dealing with sprawl.  

Primary Sources of Data Sprawl

Operational Applications:  There is often data sprawl built into many organizations because of application data redundancy. It's common to have many data sources for the main subject areas such as customers, products, services, vendors, partners, etc., in base operational data, including their archives that have been building for years or decades. As these systems struggle to keep up with change, they need additional sources of data.

Analytic Potential:  Organizations are constantly collecting data for future analysis to pick up on strategic trends, adjust tactical policies/rules, or look for operational tweaks for performance improvement. Often automation opportunities are hidden in the data generated by signals, events, and patterns occurring in typical contexts and, in some cases, divergent contexts. The wide variety of data types and context crossing requires emergent views and new data sources. Many data warehouses, data lakes, and oceans are being generated for hopefully valuable future analysis. It's complicated by new and emerging data types such as voice, image, and video.

Edge Requirements: As organizations are driven to make decisions earlier at the edge of their organizations, more immediate decisions, plus the data that support them, must be gathered. Also, data must be archived for future audits and management review of edge actions. IoT can complicate an organization's data management strategy because it drives data issues faster than traditional edge issues. Often the outcomes of edge decisions feed the analytic and operational data needs over time.

Significant Effects of Data Sprawl

Complexity:

For many organizations, data sprawl compromises the value of the data. For example, all business and technology professionals have to deal with data from multiple sources in multiple

formats, making operations and analysis difficult. In addition, data can be misinterpreted or, worse yet, corrupted during data leverage and rendering the efforts worthless or just plain wrong.

Security:

 This ever-growing data monster will be challenging to keep tabs on, thereby increasing data breaches and other security risks. In addition, it puts organizations at risk of facing strict penalties of emerging governance efforts such as GDPR, CCPA, or further data protection legislation for non-compliance.

Management/Costs:

Keeping all this data is costly and challenging to manage. The data professionals, owners, and stewards have their hands full, keeping on top of the morphing emerging data sources. All of this while assisting all the various uses of proven data, much less the data with potential whose value is unknown at any point in time.

Significant Solutions to Data Sprawl:

Shifting to the Cloud: The data discovery and classification that would occur in a cloud migration strategy would help organizations get their arms around what they have. At the same time, there would be a need to build and leverage a consolidated cloud repository where users and applications can access and store data files with ease. At the same time, silos of data can be reduced significantly by removing duplicate and irrelevant data. A Security audit can be done at the same time.

Building a Data Mesh/Fabric:  A data fabric is an architecture and a set of data services that provide consistent capabilities across a choice of data sources that are on-premises plus in multiple cloud environments. The fabric simplifies and integrates data management across cloud and on-premises data resources to accelerate long-term digital transformation while serving immediate uses. In addition, meshes/fabrics make building any data view needed easier and quicker. A significant first step in building a data mesh/fabric is acquiring a DBMS to support operational and analytical uses with the same data. It is now a real possibility that many organizations are acting on at this moment.

Building a Meta-Data Catalog: You can’t manage what you can't see or measure. It means that organizations need a data catalog with significant data descriptors for data and information resources (meta-data) that is up to date and customizable. The data discovery and classification required by cloud migration can be leveraged to build the varied catalog needed for effective data management in the digital world.

Net: Net:

We all know that getting ahead of the data sprawl is ideal where policies and procedures are in effect while gathering new data sources. Unfortunately, the reality of the day is that it's often too late for the data sources collected in the past. Organizations need to take steps now as it’s only going to get worse. We all know that new digital technologies are data-hungry, so getting them in shape for consumption is an essential business competency that needs to be grown in most cases. We also know that the hybrid work environment will generate volumes of new unstructured data to manage while change accelerates. To manage and govern our growing data sources, recent efforts around data will have to get top priority. If data is the energy source for digital progress, we have to get going now.

 

 

 

 

Results-Oriented Communications Are Now Emerging

 It is becoming painfully evident that traditional communication channels are just not making the grade in these days of group innovation and fast-moving change. Organizations are dealing with communication challenges that are accelerating in today's demanding world. If you have ever had to manage, influence, guide, or participate in or with groups of various people inside or outside your organization, you know how severe communication issues can be. These issues can only be solved by communications focused on results while all participating can be moving towards results in synchronization. The shift from messages only to include pertinent data tracked to results is the best way to find the straightest line to desired outcomes.

The Need for Outside-In Perspectives

Today customer, partner, community, employee’s needs are becoming more prominent in the most successful organizations. The outside-in approach introduces new sources of communications that will be necessary for organizational progress and success. However, it adds more complexity to an already overtaxed and unfocused set of communication channels, methods, tools, and techniques. 

The Need for Laser Focus

The world will not wait for organizations to untangle their communications messes that are growing in complexity by adding more communications on older point-to-point capabilities like email, messaging, voice, project management, and broadcasting tools. Unfortunately, none of these channels is guided by a common set of expected results or goals, leading to a lack of momentum and focus. Often the results are surprise deliverables that the sponsors did not desire or expect. All of this because of weak and unfocused communications and lack of consistent visibility.

 The Need for Broad Skillsets

The overwhelming need for better customer, employee, and partner journeys with shared goals, policies, and guardrails drive more multi-disciplinary teams that break down organizational, skill, and digital domain silos. Of course, specialized skills and training will still be needed, but lateral thinking and rich sets of compound skills assisted by digital bots or knowledge agents will be the norm.

The Need for Innovative Dynamic Experimentation

A world driven by "do it, try it and fix it" is putting additional pressure on the communications infrastructures that exist in organizations today. With iterative methods being all the rage and new digital technologies that deliver incremental results during development, communications take on new importance for keeping all the team members on point. Change is only accelerating and continues to compound communication issues.

The Need for More Collaboration

All of the above puts tremendous momentum in and around collaboration. It is why point-to-point collaboration tools have taken off, and group video tools are all the rage. Unfortunately, not all the team members can be available to keep themselves in the loop, and they will have to scramble to stay pertinent and on point. Imagine a result-oriented approach that prioritizes the critical content and messages under one roof.

Net; Net:

The need for speedy and innovative solutions is driving organizations towards results-oriented communication infrastructures that share meta-data, data, and traditional messaging methods. We are moving away from unguided point-to-point communications that create tangles and confusion to result-oriented communications that share messages, content, and data under a unified infrastructure—all of this synchronized communication leads to a shared pathway to results. The days of "willy-nilly" communications are numbered. Watch this space for more on results-oriented communications, especially in this hybrid work environment ushered in by COVID.

 

 

 

Leveraging Hybrid Cloud & Multi-Cloud

We all know the world is headed to the cloud because of potential cost savings and the ability to have a virtual data center under all conditions barring the sun generating an EMP wave that wipes out all earthly electronics. This post aims to decide what kind of cloud works best, but not what to move there or when. A common belief is that a disaster-proof dynamic cloud environment is a better option than owning and feeding an expensive and inflexible "on-prem" data center. Typically this is where Hybrid Cloud and Multi-Cloud come into play.

Hybrid Cloud:

A hybrid cloud is a solution that combines a private cloud with one or more public cloud services with software that enables the communication between each distinct service. A hybrid cloud is a powerful approach because it gives businesses greater control over their private data. Here are some of the potential benefits of hybrid cloud:

·       Better support for a remote workforce

·       Reduced costs

·       Improved scalability and control

·       Increased agility and innovation

·       Better business continuity

·       Enhanced security and Risk Management

·       Reduces the need to manage multiple vendors or platforms

A recent case study was delivered using a hybrid cloud approach. A large airline decided to build a new passenger self-rerouting to be used during difficult weather situations primarily. The new application was running in the cloud in limited use, while the remainder of the reroutes were handled via a legacy on-prem application. Along comes a problematic hurricane season, and the airline decided to roll the pilot out during brutal regional storms. Because of hybrid cloud, this was an easy, and quick switch completed successfully.

Multi-Cloud:

Multi-Cloud is a strategy where an organization leverages two or more cloud computing platforms to perform various specialized or general tasks. Organizations that do not want to depend on a single cloud provider are attracted to Multi-Cloud. Since it is not good to rely on one cloud provider, many organizations choose to use resources from several providers to get the best results from each unique service. Having multiple cloud environments ensures that you continuously have computed resources and data storage available to avoid downtime. Multi-Cloud is often a key piece in governance, risk management, and regulation compliance.

Gaining Leverage:

While there are many uses for combining these two basic cloud strategies, I believe the best leverage is creating a data mesh managed from the cloud that handles all kinds of data and utilizing views of the data quickly. The mesh means combining real-time stream data, transactional data, and archival data to serve both human needs or process or application needs. The mesh buffers the user from caring that the data is fast or slow, structured or free format, or used for analysis and business events. Good data mesh software manages the data utilizing all cloud infrastructure plus specialty features leveraging "in memory," and multi-format data at various speed ranges distributed, node focused, or centralized. All of this, with fantastic fail-over and recovery characteristics creating fault-tolerant data views or sets.

A robust data mesh can enable "fast boards" for better corporate performance or process/application monitoring. With the help of watchful human eyes, dashboard monitors, triggers, tolerances, and knowledge bots, management can stay on top of both known and emergent situations, decisions, and appropriate actions. It does this by going across data silos without cumbersome API-dependent data lakes, going across periods, and closing the gaps on various data speeds. Imagine corporate performance that can trend from the oldest archives to near-instant emergent business events and patterns, making informed decisions and taking appropriate human or system interactions in a highly optimized time frame.

While the are examples of siloed fast board approaches documented by several real-time data vendors, I expect the notion of management cockpits that span multiple areas of visibility supported by both hybrid and multi-cloud to emerge in the coming soon.

Net; Net:

This new kind of data power provided by the data mesh is only possible by leveraging all kinds of cloud resources combined with specialized "on-prem" data sources until it’s all in the cloud.

 

 

Convergent Data is Here to Stay

 The days of relying on one simple data source in any digital solution are numbered. There is a growing wave that combines multiple sources and types of data to maximize business results. Those organizations that ride this wave will thrive and capitalize on the changing conditions and emerging business moments. The best digitally-enabled organizations will leverage convergent data in new ways to stay relevant in their current business models, extend existing business models or invent new business models. This post will concentrate on the many tributaries to this evolving convergence.

              Figure 1 Convergent Data Sources

Reasons for Convergent Data

Convergent data is a result of both push and pull trends. There is the push of technological progress that invades organizations with capabilities and results that are hard to pass up. As success breeds more success these new sources of data such as video, voice, and integrated video over GPS data, organizations are hard-pressed not to leverage some if not all of these new data sources. The pull comes from the increasing demand for deeper understanding represented by multiple traditional sources combines with new and progressing data sources. In other words, the problems demand more complicated and complex data sources.

Leveraging Convergent Data

All aspects of running an organization will need or even require convergent data. Convergent and more complete data sourcing helps organizations sense events or patterns of interest, define opportunities or threats, ideate potential responses, and experiment with or implement proper responses. Each step in design thinking requires richer data offered by convergent data sources. 

Sources of Convergent Data

Refer to Figure 1 for an illustration of the typical sources of convergent data. Starting with the bottom of the chart with base data, moving up to the time continuum, and finally to the top, base data is organized into more functional groups of related data to save time and effort.

Base Data:

Operational Data is typically at the lowest level and considered as base data. Its domain definition is usually defined, such as text, numbers, etc., and relationship to other data.

Video/Image Data is typically a picture represented by pixels captured at a point in time representing positions, colors, objects, texture, and relative positions of multiple items to each other. Video is a set of successive images captured over time to create movement in time and space.

Voice Data is sound captured over a time continuum to represent inflection, volume, anomalies, and emotion to leverage in the capture, storage, and analysis for sentiment and reoccurring themes/events.

Aged Data:

Instantaneous Data is the date that is captured at the moment of focus. It is often associated with a real-time signal, event and is often more usable as a pattern, especially from multiple sources. This kind of data is often related to the Internet of Things (IoT) but not exclusively.

Archival/Backup Data is older than in nature and deemed necessary to capture in a specific time corridor. It would be ready and prepared for backup data to replace current data in a short period to repair data loss. Most archival data kept for historical purposes and trend analysis.

Organized Data:

Aggregated Data is base data grouped and organized in a fashion deemed useful for further processing or analysis. Often this data is from multiple sources and is often summarized for ease of use and access.

Management Data is data that helps organizations manage better at several levels. The most frequent service would be for describing operations to monitor or optimize operational activity. While less common, but more important would be to adjust management policies for better organizational outcomes. For ultimate impact, data would be leveraged to adjust major strategies to adapt to emergent threats or opportunities.

Time Series Data is data captured in predetermined time slots and quite often organized specific sequences.  Typically this is detailed data, but it could be summarized or aggregated in other categories.

Net; Net:

Organizations will either ride the convergent data wave, which typically creates views across multiple data sources, or get swamped by it. Convergent data is an unstoppable trend that promises new revenue sources, better situational awareness for operational optimization, and better customer engagement sources. The challenge here is managing all this convergent data smoothly. There are emerging methods, techniques, and technologies ready to assist this portion of digital transformation.