By Pengyuan Zhou and Lik-Hang Lee for 360info
Many people believe that technology is neutral or unbiased, but the reality is more complex, especially in an immersive environment.
If the proponents of the metaverse are successful, we will one day be lining up for health care or a mortgage in a virtual world ruled by virtual decision makers. The design of the artificial intelligence systems that drive this world, still the task of humans, has a real potential for harm.
In addition to commercial incentives, implicit biases that exist offline based on ethnicity, gender, and age are often reflected in big data collected on the Internet.
Machine learning models trained using these biased datasets unsurprisingly adopt these biases. For example, in 2019, Facebook (now Meta) was sued by the U.S. Department of Housing and Urban Development for “encouraging, enabling, and causing” discrimination based on race, gender, and religion through its advertising platform.
Read | Metaverse may be another “hype” like AR/VR that failed to hook people
Facebook later said it would take “significant steps” to end such behavior, but it continued to provide the same discriminatory advertising service to more than two billion users based on their demographic information.
Technical flaws in data collection, sampling, and model design can further exacerbate unfairness by introducing outliers, sampling bias, and time bias (where a model performs well at first, but fails in the future because future changes were not taken into account when building the model).
As AI increasingly invades our daily lives, governments and tech giants have started talking about “trustworthy AI,” a term formalized by the European Commission in 2019 with its guidelines on AI. of ethics.
The guidelines address issues of fairness, but current systems are already challenged to define what is fair on today’s internet, let alone the metaverse.
A recent study exploring trustworthy AI and the metrics selected to deliver it found that most were based on feature-centric design as opposed to user-centric design.
With specific regard to search engine ranking and recommendation systems, we already know that search engine rankings sometimes systematically favor certain sites over others, distorting the objectivity of results and losing the trust of users.
In recommender systems, the number of recommendations is often set to promote products or advertisements with greater business benefits instead of fair recommendations based on the ethical use of data.
To solve these problems and provide “trustworthy AI”, search engines must ensure that users receive neutral and unbiased services. Deciding on equity parameters is where it gets tricky.
A common metric selection strategy is to focus on one factor and measure how far away from equality that factor is.
For example, for search engine rankings, focus on the (potential) elements of attention received from users in terms of factors such as click-through rates, exposure, or inferences from content relevance. And then working on the gap between what an average user sees and where there’s a bias at play.
Reviewers of recommender systems have used similar measures for fairness, such as bias, mean, and score disparity.
Reliable AI design and selection of metrics in such systems also often focus on functionality during specific lifecycle phases. Ideally, it should consider reliability throughout the life cycle of use.
These considerations will be even more important in the metaverse. Immersive in nature, the metaverse is more tied to the feelings and experiences of users than current cyberspace. These experiences are more difficult to quantify and evaluate, and pose more challenges to those trying to determine what “fair AI” is.
The current mindset of reliable AI design and metrics selection, constrained by the aforementioned design philosophies, considers only part of human cognition, specifically the conscious and concrete domains. which can be more easily measured and quantified.
Pattern recognition, language, attention, perception and action are widely explored by AI communities.
The exploration of unconscious and abstract domains of cognition, such as mental health and emotions, is still new. Methodological limitations are a major reason for this, for example the lack of devices and theories to accurately capture bioelectrical signals and infer someone’s emotions.
A new set of metrics will be required for the metaverse to ensure fairness.
Select data carefully. Simply throwing data at an AI model is dangerous: data often inherits bias from the real world where it was collected. System operators should carefully select data samples focused on ensuring data diversity.
Design a fair system. The system must ensure that all users have a neutral use and are not influenced by factors such as age, education level, environment, etc. Equitable system design can help ensure diversity in data collection.
Design a fair AI algorithm. Aiming to improve the usefulness of the majority, AI algorithms normally prioritize optimizing common performance metrics such as accuracy.
For this reason, many AI algorithms establish thresholds to avoid the participation of users who may impact this goal, such as those with poor networks. Balancing the tradeoff between algorithm performance and fairness is important in designing a fair AI algorithm.
Ensure fair use. After designing a fair system and algorithm and training with fairly collected sample data, the next step is to ensure fair use for all users without bias based on ethnicity, gender, age , etc.
This last element of the cycle is the key to maintaining fairness by allowing the continuous collection of various data and user feedback to optimize fairness.
(The authors are respectively associated with the University of Science and Technology of China and the Korean Advanced Institute of Science and Technology)