Designing Decisions: Process – Part 2

In my previous post, I tried to define Decision Intelligence and the role of AI. In this post, I focus on the approach, use cases and process components. There are 4 – 7 types of AI classifications based on structure, behavior, capabilities, functions and principles (Professor Pei Wang, Temple University, PA). They are classified into two major use cases

AutomationAugmentation
Replacing HumansSupporting Humans, Human Centered
Eg. Warehouse Logistics, Manufacturing – production lines, Translations, Transcription, Customer Support Chatbots, Screening Job applications etc.Eg. Medical analysis to identify treatment options, Radiology, Prediction Markets, Insurance Underwriting, Investment Banking, Credit Approvals.
Table 1: AI Use Cases

According to Professor Ben Shneiderman, University of Maryland, besides the dangers of excessive human control and excessive automation, we are able to create a more reliable, safe and trustworthy system by balancing between Automation and Augmentation (Human Control and Computer Automation). Artificial intelligence has been discussed & practiced for the past 60 years and it will take another decade for the current technology to catch up to the IQ of a 5-year-old child. However, understanding the AI transformation journey towards automating sum or parts of the system, influence the road-map and the quadrant (Figure 1) we want accomplished. Unlike software, AI applications need time to learn from the errors and improve accuracy by tuning the models over time. This tool may give us a strategic direction of the intended AI behavior, but what other components constitute to a holistic process?

Figure 1: Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy | Ben Shneiderman (2020) Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy, International Journal of Human–Computer Interaction, 36:6, 495-504, DOI: 10.1080/10447318.2020.1741118

The Process

Based on the component model representation, (PDF) from the analysis of 280 Human AI Guidelines (Figure 2), It is logical to start by analyzing the AI behavior when you already know the problem. However, in order to solve the right problems, I would argue that the missing piece is

  • The research to understand the heterogeneous ecosystems (decision makers, influences, user groups, vendors etc.), strengths, weaknesses, data strategy and pain points around existing business process.
  • Strong guidelines to an approach that is more centered towards life on land

Before anyone knew a women is pregnant, Google or Amazon does. AI is being utilized to extract human emotions for use in in-ad content, reward to engage people as long as possible (Figure 3), use it to create fake news, revolutionize content, amplify the dissemination of stories about a specific demo graphic, Enterprise systems enact on biased data or governments with no prior research of ecosystems (Eg. Chinese Social Credit Systems) . Do we understand what phenomenon we design in a heterogeneous ecosystem? What are the costs of project failures to life? In the Kuhnian sense, maintaining egalitarianism by measuring outcomes of all connected agencies at play helps sustain living conditions. By studying the intra-actions of human and non-human actors within an ecosystem, help understand, manage and make decisions within connected ecosystems. The phenomenon of Covid is a remarkable illustration of how the use of collective intelligence has sparked numerous online partnerships, international movements, and hackathons.

Figure 2: Component Model Representation of Human-AI Guidelines | Hariharan Subramonyam, Jane Im, Colleen Seifert, and Eytan Adar. 2022. Solving Separation-of-Concerns Problems in Collaborative Design of Human-AI Systems through Leaky Abstractions. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 481, 1–21. https://doi.org/10.1145/3491102.3517537
Figure 3: Jake Tapper’s Tweet

The MIT Center for Collective Intelligence’s director, Thomas Malone, asserts that by hyper-connecting small groups of people with general intelligence to one or more computers with specialized intelligence, we create a collective super intelligence or supermind. (figure 3) He further classifies supermind’s group decision making into 5 types

Figure 3: Super minds
HierarchiesDemocraciesMarketsCommunitiesEcosystems
Decisions are hierarchical (top to bottom) and delegated to individuals in the group Decisions are often made through VotingDecisions being made between buyers and sellers. mostly single ecosystemsDecisions are made by establishing a tacit agreement based on shared standards and reputationLaw of the jungle and survival of the fittest guide decision-making. Connected Ecosystems
Table 2: Types of Superminds

By understanding an organizations business model & the heterogeneous ecosystems around it, we can create systems of change. So, how do we approach this?

According to the Delta Model by Professor Hax at MIT Sloan, Strategically there are 3 types of business approaches to a product. An opportunity exists by questioning the type of strategic approach we want AI to solve. For instance,

  • How do we make sure that everyone is a part of the processes that create the world?
  • Increased use of monitoring, significant security breaches, and other things are already happening. How can the modern connected world be “regulated”?
  • How do we increase the sense or purpose and participation?

Figure 4: The Delta Model
A Best Product Positioning PlayerTotal Customer Solutions PlayerSystem Lock-in / Network Externalities
You create the best product in the industry by having AI with the best technologyA product/service that incorporates all the features & solutions required for the customerBuild a large user base or ecosystems that work for the business.
Table 3: Types of AI Business Approaches

While, Decision Intelligence Design is more a branch focused on Data and Analytics domain, The process of solving a problem through AI is the same. I have attempted to capture them together through the process components below.

Possible RolesProcess ComponentsTools
UX Designer/ AI DesignersBusiness Process Models:

Strategic Approach:
How will AI play out in a long-term sustainable advantage?
What are the advantages to the other lives on land?
Operational Approach:
What are the sum or part of the business process that AI will intervene along with other collateral’s and assets?
Ecosystem Needs around business:
Service Design Tools
Ethnographic Research
Ecosystem Maps
UX Designer/ AI DesignersHuman Mental Models:

Task Model:
How people perform tasks today?
What are their needs and challenges?
Expectation Model:
How to set expectations about what AI can do and cannot do to achieve task goals?
Interaction Model:
How people might want to invoke AI to achieve their task needs?
User needs of AI behaviour:
User Log Reports
Labeled User Data
User-friendly Model Outputs
Storyboard with AI Interactions
User needs of Training Data:
Qualitative Codebooks
Structured Template and Data Patterns
Surveys
User Segmentations
AI Designers/ ResearchersAI-powered User Interface:

Input/Output:
How to align user inputs with AI needs?
How to present AI results to users?
Explainability: 
How to support human understanding of AI results?
Feedback:
How to design the interface so that users can provide inputs for AI to learn?
Failure/ Handoff:
How to display error and provide paths from failure?
How to design handoff when users need to pick up from AI?
User feedback for iterative AI design:
Videos of user testing
Direct feedback from users
Engineering participation during user testing
Data Science EngineersAI Models

Design:
How to design automation around human needs?
Model Performance: 
How to ensure accurate AI performance for diverse users and usage scenarios?
Evaluation:
How to define success metrics for AI?
How to evaluate AI with users?
Learnability:
How to design for co-learning and adaptation?
AI Implementation for Human Centered Evaluation:
Model outputs, features and weights
Knobs to tune model parameters
Greybox prototypes
Model rules and assumptions
Model logic visualization
Model behaviour for UI/UX design:
Function logic/ API annotations
Raw model outputs
Dashboard for AI performance
AI capability demo prototypes
Data Science Engineers & AI DesignersTraining Data

Needs:
How to plan dta needs around human needs?
Collection: 
How to collect diverse and unbiased data that reflects the context of use?
Labeling:
How to determine labels that align with human needs?
Privacy:
How to design privacy features for personal data and human trust?
Data characteristics of UI/UX design:
Dataset Specifications
Raw JSON Data
Computational Notebooks
Table 4: Modified Component Model & tools from the research paper “Component Model Representation of Human-AI Guidelines” (Figure 2)

These process components can be aligned to design approaches like design thinking, agile design and participatory design. I will try to attempt that in the next post.