The Effect of Video Game Artificial Intelligence on Player Satisfaction by Drew Ackerman A thesis submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE OF COMPUTER SCIENCE WEBER STATE UNIVERSITY Ogden, Utah December 2021 _ _______________________ (signature) Brian Rague Committee Chair ________________________________ (signature) Robert Ball Committee Member ________________________________ (signature) Richard Fry Committee Member ________________________________ (signature) Drew Ackerman A Thesis in the Field of Computer Science for the Degree of Master of Science in Computer Science Weber State University December 2022 Copyright 2022 Drew Ackerman The Effect of Video Game Artificial Intelligence on Player Satisfaction Abstract As major video game developers seek broader audiences to sell their product, video game participation has grown quickly in the last decade. Most games developed include an artificial intelligence (AI) system to control characters that interact with the player. This requires designers and developers to consider the impact they want the AI to have on the players experience. This project became possible to develop as video games become more advanced and the capabilities of visual computing and interactive medium grew. The growth the gaming industry has experienced in the last decade opens new areas of research for figuring out the best use of a video game developer’s limited budget and time. This thesis will examine the relationship between a video game’s adaptive AI performance and player’s satisfaction. A traditional maze action game was developed by the researched to gather data for this paper. Additional systems were developed to monitor the player’s playtime, gather demographic and performance data, store the data, and analyze the data. Acknowledgments To my wonderful partner and family for helping keep me on track and encouraging me to progress on this project. Thank you to my board for their endless patience and in helping me with this project. Thank you to my aunt for expressing a profound importance in continued education in any capacity. Taking this difficult journey into a master’s program is because of you. Table of Contents Acknowledgments..............................................................................................................iii Introduction.........................................................................................................................6 Related Work.......................................................................................................................7 Methodology.....................................................................................................................11 The Hypothesis......................................................................................................11 The Game...............................................................................................................11 AB Testing.............................................................................................................15 Enemy AI...............................................................................................................15 Actions...................................................................................................................18 Chase..........................................................................................................19 Ambush......................................................................................................19 GoTo..........................................................................................................20 Scatter........................................................................................................20 AI Mutation and Culling........................................................................................21 AI Culling..............................................................................................................22 GoTo Bug...............................................................................................................23 Levels.....................................................................................................................23 Player Participants.................................................................................................24 Data Collection......................................................................................................27 Data Management..................................................................................................27 Data Collection......................................................................................................27 Results...............................................................................................................................29 Demographic Statistics..........................................................................................29 Graph Analysis.......................................................................................................31 Survey Analysis.....................................................................................................36 Conclusion........................................................................................................................42 Bibliography..........................................................................................................43 Appendix A Consent Form................................................................................................46 Appendix B........................................................................................................................50 “The Appointment in Samarra”.............................................................................50 Introduction As a form of entertainment, the video game’s main purpose is to provide an enjoyable outlet to the player. This objective can be archived in many ways, as different players seek different things. For example, some players enjoy a competitive aspect to their games, while others are searching for a story. Most games, regardless of the type, need to implement an AI system. The AI system expands a static game into a dynamic one. A dynamic AI attempts to provide an optimal experience by understanding the player’s temperament and strategy and then appropriately challenging the player. This paper’s objective is to determine if there is a correlation between adaptive AI and a player’s satisfaction in a video game environment. The related works section will explain the research that came before this paper and explain the purpose of this study. The methodology section will cover the implementation of the game used during the study. The results section will discuss the data gathered from the studies participants. The conclusion section will explain the implications the results have on the study and suggestions for future works. Related Work Since the emergence of the video game several decades ago, there have been numerous studies done concerning the effects and interaction between a video game and the effects on the player. In a study by Kahn [6] six categories were created to help determine the motivations of video game players. The motivations listed in Table 1 shows that players have a variety of incentives for why they play video games. Understanding these motivations could be used to create a better AI or gaming experience. Table 1 Player Motivations Player Motivation Socializers Completionists Competitors Escapists Story-Driven Smarty-Pants Types of player motivations Johnson [5] attempted to correlate player variables with increased play time. Several variables, such as gender or game genre, had a strong correlation with increased play time. 8 Table 2 Flow Attributes Flow Attributes A Challenging Activity That Requires Skills The Merging of Action and Awareness Clear Goals and Feedback Concentration on the Task at Hand The Loss of Self-Consciousness The Paradox of Control The Transformation of Time Attributes commonly present in games with Flow. Flow, as described by Csikszentmihalyi [2], allows “a sense of deep enjoyment that is so rewarding people feel that expending a great deal of energy is worthwhile simply to be able to feel it.” Flow can be achieved by a combination of the following activities listed in Table 2. A study by Sepehr [9] found that Flow has a large impact on a player’s satisfaction. Additionally, they found that Flow was correlated to arousal, with arousal being affected by challenge. Heightened arousal influenced how the participants paid attention to the activity, which caused higher engagement and an experience of Flow. Table 3 Satisfaction Attributes Satisfaction Attributes Uncertain Outcome Variable Difficulty Multiple Level Goals Score Keeping Hidden Information Randomness Decoration Rewards Engage Curiosity Constructive Feedback Attributes attributed to higher satisfaction. 9 In a summary of his previous work and others, Malone [8] discusses the attributes that computer games should express to increase satisfaction, these attributes are listed in Table 3. Adaptive AIs are researched in Tan [11], Swiechowski [10], Gilleade [3], Ibanez [4], Kuan [12], and Arzate Cruz [1]. In Tan [11] adaptive behavior-based AI was used to scale difficulty in a game. The paper identifies that an adaptive game AI should express unpredictable but rational behavior while also profiling a player’s game play style to produce an interaction that is rewarding and entertaining. The AI contained behavior-based components and tactical components. The tactical component of the AI plans and decides what goals to achieve. The behavior component determines how the AI achieves that goal. An AI drive by a Utility System is explored by Swiechowski [1] to build a framework that could be used for adaptable AI in video games. The utility system allows consideration, various aspects of game state, to be chosen and then a utility function defined that can be used to compare against other utilities or groups of utilities. This system is then utilized in an evolutionary algorithm to promote change. Gilleade [3] proposes an adaptive video game that changes based on a players frustration level. An experiment by Ibanez [4] developed an adaptive AI and recorded the participants feelings of difficulty, frustration, boredom, engagement, and willingness to play again. Two players would play against one another. The AI adjusted game play to make a game more difficult or less difficult depending on the individual players ability in the game. It was found that when the adaptive AI was introduced into game play that 10 both players would have an increase in engagement, and the willingness to play again, while also experiencing less frustration, boredom, and a sense of difficulty from the game. Kuan [12] created an adaptive AI for players of the Gamoku game. The AI had 8 separate levels of difficulty. The AI could adjust its difficulty depending on how well the opponent was doing during game play. If the AI determines the opponent made several consecutive best moves possible then the difficulty level increased. If the opponent repeatedly makes the optimal move, then the Ais level decreases. The study concluded that participants were satisfied with the difficulty level of the AI as the AI adjusted during game play. Arzate Cruz [1] describes how a player-centered game AI functions by adjusting key game features where each feature is separate, but all features share the same player representation module. Each game feature utilizes the player representation to adapt their individual features regardless of other game function states. Arzate Cruz [1] suggests an implementation of dynamically scripted AI with the following attributes: Player typification, the ability to apply a challenge, feature selection, optimization algorithms, and constant difficulty curves. Lucas [7] summarizes currently available techniques for training an AI agent. The recommended techniques are Evolutionary Algorithms, Evaluation and Competition, and Reinforcement Learning. 11 Methodology The Hypothesis This research attempts to answer the hypothesis, an adaptive AI has no effect on player satisfaction. The Game The game was developed by the researcher. Art and music assets were provided by collaborators and colleagues of the researcher. To test the hypothesis, it was necessary to design a game that facilitated the following: 1. Simple controls 2. Easy game play for all participants of the study to quickly understand 3. A simple enemy AI that could be changed quickly and easily Emulating the 1980’s arcade game Pac-Man would satisfy all three design conditions. The simple controls condition is fulfilled by the player only having four actions available; move north, south, east, or west. Participants were able to quickly understand the game because of the simplified design and the small user guide that was accessible from the main menu. Pac-Man’s simple AI was turned into discrete actions, that when combined, allowed complex enemy AI behaviors to be expressed. A theme was chosen to increase player interest in the game. The theming of the game was based on the short story, “The Appointment in Samarra”, which can be referenced in appendix B. Because of the chosen theme, the player is a merchant, 12 collectable scoring items are gold and gems, and the enemies are the Four Horsemen of the Apocalypse. The enemy names are Conquer, Death, Pestilence, and War. The different enemies are easily identified by their color. Conqueror is white, Death is purple, Pestilence is green, and War is red. The Unity Game Engine was chosen as it provides a series of components and tools that simplified the development of the game. Unity was chosen over other alternatives as code can be written in C#, a language that the developer was familiar with. Microsoft Visual Studio Pro was used as the Integrated Development Environment (IDE) because it easily integrates with Unity and provides expert tools for developing C# code. Figure 1 Maze in the game The maze that players navigate 13 The game is played on a 2D grid-based maze that is 39 tiles wide and 22 tiles high (See Figure 1). Every walk-able tile on the grid has a collectable object. Power-up collectables and pellets are not respawned during the level. Bonus objects are respawned during the level. Pellets are a score and level completion mechanic. When collecting pellets, the player is rewarded with an increase in their current score, this sub mechanic helps drive the player to complete the level and progress in the game. A level is completed when all pellet collectables are gathered by the player. When a level is completed the level resets. A level reset is accomplished by completing the following: 1. Increment level counter 2. Mutate and Cull enemy AI, if AB testing is not occurring 3. If the player has a new high score, save it to the server 4. Update the players total play time 5. Place Player at Player Spawn, south of the Graveyard 6. Set the Players lives to three 7. Despawn enemies, reset their activation counters 8. Spawn power-up collectable, and pellet collectables. Every 30 seconds a bonus collectable is created and placed on 1 of 6 possible tiles. The bonus collectable lasts for 15 seconds before destroying itself. The value for picking up the bonus collectable is explained by the following algorithm Value = 500 * CurrentLevel Where CurrentLevel is an integer representation for how many levels the player has completed, within the range of [1, MaxLevel]. 14 The player can move in the four cardinal directions: north, south, east and west, along designated pathways. Figure 3.1 shows the walk-able paths. The player can change direction at intersections in the maze if a path is available. The player can reverse their direction at any time. The player must avoid four enemies while navigating the maze. The enemies spawn in the center of the maze. This spawning point has an exit at the north. Enemies may only exit this spawn point; they cannot enter it. Players cannot enter the enemies spawn point. Figure 2: The Enemy Spawn Point The graveyard potion of the maze where enemies spawn. The enemies spawn from their statues. Figure 2 shows the location where the enemies spawn. This enemy spawn point is not accessible by the player. Enemies are slowly released when the player has gathered a certain number of collectables. When the enemies spawn at their corresponding statues, they proceed to the north exit and then begin navigating the maze. If an enemy captures the player, then the player loses a life and the level resets. A player can interact with four power-up objects placed in static locations in the maze. 15 When the player collides with the power-up they become invincible for five seconds. During the invincibility period the player can kill an enemy by running into them. When an enemy is killed, they respawn back at their starting spawn point. A player has three lives before the game ends at which point the player is given a chance to play the game again. AB Testing This study was designed for each participant to undergo AB testing. The objective of the AB testing was to see if players would score the game differently during surveys because of a perceived impression that the AI was evolving. The two states of play were A-Changing AI Session, and B-Default Testing Session. During normal game play, the enemies’ AI would mutate and cull when the player completed a level. This AI would then be saved and used in later level. During B-default testing the enemies are given a default AI. The default AI does not change between AB tests. Enemy AI Each enemy is assigned a default AI at the start of the game. Death’s default AI is an aggressive chase AI. The other three Horseman’s Ais are pseudo-aggressive Ambush AI. The difference between the aggressive and pseudo-aggressive AI lies in the effect they can have on the player. The idea being that an aggressive AI will be relentless in pursuing the player, which provides no chance for the player to rest. A pseudo-aggressive AI will still chase the player, but also provide a bit of space, which gives the player breathing room. 16 Figure 3 Cyclic Directed Graph Structure An example of a cyclic directed graph. The AI is implemented with a directed cyclic graph structure. An example of a directed cyclic graph structure is shown in Figure 3. A directed graph allows traversal of an edge in one direction. In Figure 3, Node 2 can traverse an edge to Node 3, but traversal is not allowed from Node 2 to Node 0. The direction of travel along an edge is shown via arrows. Directed edges were chosen to encourage the AI to choose different paths through the AI and to prevent the AI from only traversing between a pair or small group of actions. The cyclic property of the graph allows nodes to contain edges that start and end at the same node as well as create a loop via a series of edges. The cyclic behavior is shown on Node 1 in Figure 3. 17 Table 4 Adjacency List From To Node 0 Node 0 Node 0 Node 1 Node 0 Node 2 Node 1 Node 3 Node 2 Node 3 Node 3 Node 0 Adjacency List for Figure 3.3. Each node or vertex in the graph contains an Action. An adjacency list holds the inbound vertex and outbound vertex for all edges in the graph. Each node can have multiple adjacencies. The graph’s edges are held in an adjacency list. The adjacency list as shown in Table 4, defines the edges entering and leaving the node. For example, From Node 1 and To: Node 2 describes an edge that starts at Node 1 and ends at Node 2. When an action at a particular node is used, a variable for that action is incremented. This incremented variable is used to determine how many time that node has been visited. Figure 4 Cyclic directed graph with actions A graph that demonstrates how complex behaviors can be created from a group of simple actions 18 All AIs start at the root node. When the Action of a node completes, a random outbound edge for that node is selected. Random selection was chosen as it allows complex behaviors to be expressed from a small collection of actions. Figure 4 is like Figure 3 but with action names included. This graph shows that, depending on the edge chosen, the overall behavior of several consecutive actions is different. The path Chase, Ambush, Chase, expresses a different behavior than the path Chase, GoTo, and Chase. The first path creates a behavior that is more aggressive, while the later path is less aggressive because the GoTo action inherently puts less pressure on the player. Actions Actions are the fundamental behavioral unit of the enemies. There are three aggressive actions and one defensive action. The three aggressive actions are: 1. Chase 2. Ambush 3. GoTo The defensive action is Scatter. Each action contains properties whose values are bound by unique ranges. Each action implements a heuristic function to determine how successful the action is over time. The heuristic functions for the action are explained in each actions section below. 19 Table 5 Action Descriptions Action Name Behavior Properties Design Purpose Chase Attempt to catch the player TimesPlayerKilled TimesNearPlayer TimesActionCalled Pressure the player and provide a sense of urgency Ambush Catch the player by predicting their future location TimesPlayerKilled TimesReachedDestinaton TimesActionCalled Pressure the player and provide a sense of urgency GoTo Go to a specific location on the map TimesArrivedAtDestination TimesActionCalled Remove pressure from player Descriptions of an Actions behavior, properties, and why that action was designed. Chase When the Chase action is active, the enemy will attempt to collide with the player. The chase action defines a certain amount of time that the action will last before ending and proceeding to the next selected action. The heuristic for Chase was chosen so that a pressure was applied to evolve a chase action that is more successful in killing the player rather than chasing the player. Ambush When the Ambush action is active the enemy selects a tile N tiles ahead of the players current direction of motion. The goal of this behavior is to cut off the players route and apply pressure in a similar manner to the Chase action. The heuristic for Ambush chosen to equally value killing the player as much as getting close to the player. The goal was to create an action that provided less pressure to the opponent during game play. 20 GoTo When GoTo is active, the enemy marks a tile and continually tries to get to a marked tile for the duration of the action. Effectively the tile is marked as a target, and the enemy seeks to arrive at the target within a designated time. The GoTo heuristic was chosen to value successfully arriving at the destination. This pressure should produce a GoTo action that removes the enemy and relieves pressure on the player. After the data collection portion of the project was completed, a bug was discovered in a part of the GoTo implementation. The TimesArrivedAtDestination property of the GoTo class is not properly tracked. The original implementation meant to use the property as an integer that was incremented when the AI agent arrived at the destination, essentially measuring the number of times the enemy successfully completed its GoTo action. This property would then be utilized during the AI mutation and culling portion of the AI. This bug will be further explained in section GoTo Bug. Because the chance a player encounters this bug is influenced by the randomized factors of mutation and action selection, the severity of the effects on the user varies. The ultimate affect this has on participants is that the AI has the potential to, overtime, become more aggressive than expected. Scatter When the player picks up an Invincibility power-up, all enemies cancel their current action, and a scatter action is applied. Enemies will reverse direction from the player, attempting to get as far away from the player as possible. At intersection, the enemy will look at the available directions and choose the direction that will get them 21 immediately further away from the player as possible. The scatter behavior thus acts like a reverse chase behavior. Enemies under the scatter behavior are also slowed. Slowing down the enemies allows the player an increased chance at catching enemies in their vulnerable state. An additional flashing state is given to all enemies that are currently scattering. The flashing state indicates to the player that enemies are currently vulnerable. AI Mutation and Culling When a player completes a level and B default testing is not occurring, the AI for each enemy is mutated. A mutation on the graph begins by selecting a single random node, where each node is equally likely to be selected, from the graph. A new node, N is created and connected to a random node in the graph via an edge. A new Action is created and assigned to node N. Then, a final edge is created between the node N and the root node. This mutation process is completed at least once and can be repeated up to three more times per level completion. When a new action is created during mutation, the following occurs: 1. Select a number in the set [1,2,3]. a. If the selected number is 1, then a new Chase Action is created i. Randomly select an integer between 10 and 30 inclusive. ii. Assign this integer to the Chase Action’s ChaseDuration property. b. If the selected number is 2, then a new GoTo Action is created i. Randomly select a walk-able tile on the grid. ii. Assign this tile to the new GoTo’s FinalDestination property. c. If the selected number is 3, then a new Ambush Action is created 22 i. Randomly select an integer between 1 and 10. ii. Assign this integer to the Ambush Action’s AmbushDistance property. AI Culling Culling of the graph occurs after the graph mutation step completes, regardless of the total number of mutations that occur during that step. The culling process only occurs once and only occurs after the mutation step completes. The culling algorithm will only occur on a graph that contains more than five actions (nodes), and at least five actions must have been visited more than five times. This condition prevents over-culling of graphs that have not had enough play time to begin to develop. Figure 5 Edge propagation A visualization of edge propagation. The left image is a graph before a node is removed. The right image is the graph after a node is removed. There is a possibility for this condition to prevent a graph from being culled until more play time has occurred. This condition preserves a base graph so that an AI is always present. If the condition is passed, then a cull occurs on the graph. The culling process begins by listing all the nodes that have been visited a minimum of five times. Then, the culling process ranks all the nodes in that list. The culling process selects the node with the lowest rank and removes the node from the graph. If there are multiple 23 nodes with the lowest rank, the algorithm will select the first lowest ranked node it encountered when creating the list of lowest ranked nodes. Outbound nodes are connected to the inbound nodes from the removed node. Figure 5 shows how edge propagation is done when a node is removed. Node ranking is determined by a separate heuristic for each Action, listed below. Rankin gC=(TimesPlayerKilled∗.6+TimesNearPlayer∗.4) TimesActionCalled Rankin gA=(TimesPlayerKilled∗.5+TimesReachedDestination∗.5) TimesActionCalled Rankin gG=TimesArrivedAtDestination TimesActionCalled The Chase, Ambush, and GoTo Actions are represented by RankingC, RankingA, and RankingG respectively. GoTo Bug The GoTo bug occurs because the property, TimesArrivedAtDestination, was not incremented when it should have been. The implications of this means any GoTo’s ranking evaluation, see Equation RankingG, will always evaluate to zero. This means that if a GoTo Action is culled, it will be culled before all other Actions. It is hypothesized that this causes the AI to become more aggressive over time as GoTo actions are culled before Chase or Ambush actions will be culled if all actions are equally available to be culled. Levels 24 A level is completed when players gather all collectable gold pieces on the maze, see Figure 6 for reference. A level also includes bonus collectables, listed in Figure 6. Bonus collectables allow players to accrue additional points over the length of a game. Figure 6 In game items The left image is the collectable gold piece. The right image shows the bonus collectables. Player Participants In order to have participants in this research study, approval was gained through Weber State University’s Institutional Review Board (IRB). The IRB process ensures this research study maintains state and federal guidelines when human subjects are involved. Participants were initially recruited from the researcher’s family and friends. Additionally, an email was disseminated among the student majors at Weber State University’s School of Computing. This email asked prospective players to participate in a master’s thesis research study by playing a game for at least two hours. In return all participants that played the game for at least two hours received a $15 gift card. The top 25 three scoring participants also received an additional gift card: $75, $50, $25 respectively. Communications with participants occurred via email. Email addresses were gathered from participants during account creation. The email addresses were stored on the password protected server to protect player’s identity. Each study participant had access to a public facing web address so that they could participate whenever and wherever they wanted. The web address served an HTML 5 version of the research platform game. Before playing the game, the players would need to create an account that they could then sign-in with. Players were first asked to acknowledge an informed consent that was displayed in a scrolling text box. See appendix Section A for the relevant consent form. Players could not proceed without acknowledging that they had read the consent form. Figure 7 Player Demographics Questionnaire The series of questions that players filled out when creating an account. 26 Players were asked to provide a username, email, password, and demographic information, which was stored securely to the protected database. Figure 7 shows the demographic portion of the sign-up process, where players are asked to select answers for the questions listed in Table 6. Table 6 Player Demographic Questions Question Default Selection Age 18 Gender Male Ethnicit y American Indian or Alaskan Native Country United States (US) The Questions and their associated defaults that were asked to players during sign up Only players with at least two hours of play time are considered as viable data for this study. The participants time playing the game was tracked. Tracking was accomplished by implementing a timer in game. The time was started when a level started and stopped when a level was completed. The accrued play time was then added to the user’s current play time and stored in the database. Two days after releasing the game a bug with the players playtime was discovered. This bug caused participants play time to increase even if the game was paused. This issue affected nine players. The issue was fixed by stopping the timer when the game is paused and resuming the timer when the game is resumed. Of the 155 original participants, 40 completed at least 2 hours of game play. This equates to 26% of participants being viable research subjects. In the rest of the thesis, the only statistics given will be for the qualified 40 players. 27 Data Collection Data was collected during game play, condensed into JSON, and then sent over HTTP. Data was stored in a MySQL database. Data was parsed and analyzed with Python and JavaScript. Data Management Data was stored in a MySQL database in relational tables. Unique user identification numbers were assigned to users when they created their accounts. These integer IDs were used as primary and secondary keys to allow records to be tied back to an individual user. The data was securely stored on the researchers account with a complex password. To analyze data, the database tables were converted to Comma Separated Value (CSV) files. The CSV files were then downloaded to the researcher’s secure PC for use in analysis. Originally the data analysis was done in Python. After difficulties with displaying the AI graph, the analysis implemented in JavaScript to make use of the Vis- Network library. The Vis-Network library enabled complex graph analysis. Data Collection A survey was given to players after the completion of two games. The survey consisted of four questions. The questions were as follows: 1. How difficult was the game? 2. How satisfying was the game? 3. How difficult was the enemy? 4. How satisfying was the enemy? 28 Each question had a response range of 1-5 inclusive. Each question’s value was defaulted to 3. The question’s value could be changed with a slider bar that moved in increments of 1. Before the game was released to all participants, early feedback indicted that the questions were confusing and that players had difficulty in determining if a 1 was good or if a 5 was good. Question 4 was changed with the final survey questions listed below: 1. How difficult was the game? 2. How satisfying was the game? 3. How difficult was the enemy? 4. How satisfying was it to play against the enemy? In addition, icons were added adjacent to the slider selection to better indicate what the values meant. 29 Results Demographic Statistics Figure 8 shows the ethnicity split for participants that completed the study. Of the demographic data, 35% were American Indian or Alaskan Native. Given the geographic location where most participants resided, the magnitude of this percentage seems unlikely for this ethnicity. It is assumed that during sign-up participants either left the default selection, which was ‘American Indian or Alaska Native’ or saw ‘American’ and did not change the selection. Figure 8 Player Ethnicity Distribution The ethnicity distribution of all players that completed the study. 30 Figure 9 shows the gender split for valid participants. A large percentage of players were male with 12.5% of participants being female. One individual declined to identify as either male or female. Figure 9 Participants Gender Distribution The gender distribution of valid participants. 31 Figure 10 How Often Participants Play Video Games How often participants play video games per day. Figure 10 shows the percentage of session duration for valid participants. Most participants of the study would be classified as casual gamers at two or less hours spent playing video games per day. Nine participants would be classified as moderate gamers at two to four hours of playing video games per day. Four participants would be classified as extreme gamers with four to six or more hours of video game playing every day. Graph Analysis In this study, graphs were used as the structure of the AI. The graph structure allows for the AI to grow and shrink by adding and removing Actions. This adding and removing of actions changes the way the AI behaves as the traversal of any path within the graph is selected at random. Graph analysis involves determining the difficulty and spread of a graph. The difficulty of a graph is determined by the composition of actions within a graph. Each action is given a value. The sum of the values is the Difficulty score. Table 7 shows the values that each action is given when determining the difficulty score of a given graph. A higher position difficulty indicated that the corresponding AI would be difficult and a challenge for the player to navigate. The more negative the difficulty score indicates an AI that should be easier for the player to combat. This assumption is based upon the behaviors each Action type expresses based upon Table 5. A graph with 3 Chase, 4 Ambush, and 4 GoTo nodes would have a difficulty score of 1 which is considered a relatively easy opponent. The Ai is considered easy because there are fewer actions that 32 pressure the player, and more actions that give the player breathing room. The spread of the graph is calculated by dividing the total adjacencies by the total vertices in a graph. The higher the spread of a graph, the higher the total number of possibilities that a graph can express. Table 7 Action Difficulties Action Difficulty Score Chase 1 Ambush 0.5 GoTo -1.0 Difficulty scores for each action type. When creating a visual representation of the graph, each action was assigned a color. A red circle indicates a Chase Action, orange an Ambush Action, and blue a GoTo Action. Adjacencies were indicated with lines from vertex to vertex. An arrow is placed at the end of each line to show the direction of travel for each edge. User 61 as selected for analysis because the user had a high number of graphs and surveys. 33 Figure 11 War Graph First War graph for user 61 Figure 11 is the first Automated AI graph generated by user 61. This graph is the default connected graph for the enemy, War, that every user is given when they start playing the game. The graph has three Ambush AI actions giving the graph a difficulty score of 1.5 and a 2.333 graph spread score. Figure 12 Another War Graph from user 61 Twenty-fourth War graph for user 61 Figure 12 is the twenty-fourth AI graph submitted by user 61. This graph has a difficulty of -0.5 and a spread score of 4.857. The difficulty of this graph is substantially lower than the first graph. This shows that the AI for the War enemy expressed behaviors that were less pressuring to the player. In the graph, three GoTo AIs are expressed, which should indicate an enemy with a high chance of occasionally wandering away from the player, providing a break from aggressive actions for the player. 34 Figure 13 User 61’s final War graph Figure 13 is the 142nd Automated AI graph by user 61. This graph has a difficulty of -0.35 and a spread score of 0.357. This final graph submitted by User 61 shows a graph that expresses an enemy that is increasingly less threatening-as there are more chances for the enemy to activate a GoTo action and give the player a break from aggressive actions. 35 Figure 14 A stacked histogram for a graph Players 61’s stacked histogram of all graphs submitted The graph data for user 61 was collected across multiple play sessions, with the possibility that multiple graphs were created in each session. Figure 14 shows a histogram of the graph data for user 61 across twenty-six sessions. This histogram is stacked where the count of each Action type is shown for each graph. A red bar represents the number of Chase actions in the graph, an orange bar represents the number of Ambush actions in the graph, and a blue bar represents the number of GoTo actions in the graph. An error was discovered when putting graph data into histograms. The histogram indicates that the Adaptive AI was being reset to its default state periodically. This behavior is indicated in the graph by a graph of three ambushes being represented; this includes but is not limited to, graph three, five, eight, and sixty-five. The resetting behavior seems to occur when a player started a new game session, meaning that reloading the game in any way would reset the Adaptive AI to the default AI state, instead of continually evolving. Figure 14 shows this behavior clearly. Anytime the graph regresses to three Ambush actions, it is likely that the bug occurred. The resetting bug affects the analysis to determine a correlation between surveys and graphs. The bug compromises how satisfaction is correlated to players’ survey responses, as the player’s AI is reset before every play session. Table 8 Average Graph Spread Per Enemy Enemy Average Spread 36 Death 0.3899 Conqueror 0.3898 War 0.3904 Pestilence 0.3906 All Enemies 0.3902 The average graph spread for each enemy Graph spread was calculated for every AI graph, then, the average spread for each Enemy AI was determined. Table 8 shows the average spread value for each enemy AI. The graph shown in Figure 12 has a graph spread of 0.5. Survey Analysis Figure 15 Survey Distribution The number of A Testing surveys and B Testing surveys completed by all 40 players. 37 Figure 15 shows the count of surveys that occurred during different types of testing. A total of 471 surveys were completed by 40 players who completed two or more hours of game play. 200 surveys were completed when A testing was occurring, and the Default AI was used during the game. 271 surveys were completed when B testing was occurring, and the Evolutionary AI was used during the game. Figure 16 Survey Averages The average survey score for All, A, and B surveys. In total 471 surveys were completed for analysis. Figure 16 displays the Survey Averages when considering only surveys for Evolutionary AI, Default AI, and for the collection of all surveys. 38 Figure 17 Survey Averages The distribution of survey averages for all the surveys The average score of all surveys that were completed when the evolutionary AI was active is 3.34, the average of all surveys completed with the Default AI active was 3.39 and the average of all surveys completed is 3.37. Figure 17 shows the frequency of survey averages. This data shows most of the surveys had a score between 2.5 and 3. Figure 18 Histogram for A-Testing surveys A stacked histogram of survey data for all valid users A-Testing surveys 39 Figure 19 Histogram for B-Testing surveys A stacked histogram of survey data for all valid users B-Testing surveys Histograms shown in Figures 18 and 19 were created to display the survey data. The x-axis of the histogram is the graph number, the y-axis is the survey score, which is sum of the scores of all survey questions. The survey questions values are shown in the histogram as a stacked graph. Table 4.3 shows which color corresponds to which survey question. Table 9 Survey Question Colors Survey Question Color Round Satisfaction Red Round Difficulty Orange Enemy Satisfaction Blue Enemy Difficulty Green The colors assigned to each survey question for use in the histograms in figures 18 and 19 40 The Figure 18 displays a stacked histogram of all surveys for all players recorded while A testing was active. The Figure 19 displays a stacked histogram of all surveys for all players recorded while B testing was active. T-Test were performed on the collection of all valid surveys on each survey question. An α of 0.05 was used in each T-Test. The T-Test utilized two-sample testing, A surveys and B surveys. Table 10 shows the T-Test results for each survey question. Table 10 Survey Question T-Test Results Survey Question pVal Confidence Bounds (Lower, Upper) Round Satisfaction 0.17303 (-0.27798, 0.05012) Round Difficulty 0.46573 (-0.19381, 0.08881) Enemy Satisfaction 0.19732 (-0.26886, 0.05564) Enemy Difficulty 0.98222 (-0.14555, 0.14889) The T-Test results for survey questions done on A and B surveys. Utilized two-sample testing and an α of 0.05 The results of the T-Test for each survey question means the null hypothesis cannot be rejected and subsequently this is no statistically significant difference for each question. Table 11 Survey Total T-Test Results T-Test Value Value P-Value 0.211632 Lower Confidence -0.174422 Upper Confidence 0.038732 Test Value -1.250683 Freedom 501.820014 T-Test results on the sum of the survey questions. Utilized two-sample testing on A and B surveys. Used an α of 0.05 41 Additionally, a T-Test was performed on all of the available A-Surveys and B-Surveys. An α of 0.05 was used in the T-Test. The T-Test utilized two-sample testing. The T-Test was supplied with two arrays. The first array was a collection of each A survey. The second array was a collection of each B survey. Table 11 shows the T-Test results. With a pValue of 0.211 and an Alpha of 0.05, we cannot reject the null hypothesis and therefore conclude that a significant difference does not exist. A 21.1% likelihood exists that the results were produced by chance and there is no statistically significant difference between the A and B surveys. This data accepts the hypothesis – “An Adaptive AI has no effect on player satisfaction”. 42 Conclusion This paper set out to determine that an adaptive AI has a positive effect on player satisfaction. T-test analysis done on the surveys does not allow the null hypothesis to be rejected. This means that a meaningful correlation cannot be made between player satisfaction and the use of an adaptive AI in this project. While the surveys were a great way to extract quantitative data from qualitative data, it’s difficult to ensure that previous game play experiences do not affect future surveys. Cyclical directed graphs were utilized in creating an adaptive AI with evolving behavior. This research illustrates the benefits of an Adaptive AI, but it also raises the question of how complex of an AI is needed before player satisfaction is increased. To better understand the implications of these results, future studies could address the effects of increasingly complex AI, or relationships between types of AI and their various effects on the player. 43 Bibliography Christian Arzate Cruz and Jorge Adolfo Ramirez Uresti. Player-centered game ai from a flow perspective: Towards a better understanding of past trends and future directions. Entertainment Computing, 20:11–24, 2017. Mihaly Csikszentmihalyi. Flow: the psychology of optimal experience. Harper- Perennial, New York, 1991. Kiel Gilleade and Alan Dix. Using frustration in the design of adaptive videogames. pages 228–232. ACM, 2004. Jesús Ibáñez and Carlos Delgado-Mata. Adaptive two-player videogames. Expert Systems with Applications, 38(8):9157–9163, 2011. Daniel Johnson, John Gardner, and Penelope Sweetser. Motivations for videogame play: Predictors of time spent playing. Computers in Human Behavior, 63:805–812, 2016. Adam S. Kahn, Cuihua Shen, Li Lu, Rabindra A. Ratan, Sean Coary, Jinghui Hou, Jingbo Meng, Joseph Osborn, and Dmitri Williams. The trojan player typology: A cross-genre, cross-cultural, behaviorally validated scale of video game play motivations. Computers in Human Behavior, 49:354–361, 2015. Simon Lucas, Philipp Rohlfshagen, and Diego Perez. Towards more intelligent adaptive video game agents: a computational intelligence perspective. pages 293–298. ACM, 2012. Thomas W. Malone. What makes things fun to learn? heuristics for designing instructional computer games. In Proceedings of the 3rd ACM SIGSMALL Symposium and the First SIGPC Symposium on Small Systems, SIGSMALL ’80, page 162–169, New York, NY, USA, 1980. Association for Computing Machinery. Sepandar Sepehr and Milena Head. Understanding the role of competition in video gameplay satisfaction. Information and Management, 55(4):407–421, 2018. Maciej Swiechowski and Dominik Slezak. Grail: A framework for adaptive and believable ai in video games. pages 762–765. IEEE, 2018. Chin Hiong Tan, Kay Chen Tan, and Arthur Tay. Dynamic game difficulty scaling using adaptive behavior-based ai. IEEE Transactions on Computational Intelligence and AI in Games, 3(4):289–301, 2011. 44 Kuan Liang Tan, Chin Hiong Tan, Kay Chen Tan, and Arthur Tay. Adaptive game ai for gomoku. In 2009 4th International Conference on Autonomous Robots and Agents, pages 507–512, 2009. C. Nandini, S. Kumar and R. Laasya, "AI Poker Agent based on Game Theory," 2019 1st International Conference on Advanced Technologies in Intelligent Control, Environment, Computing & Communication Engineering (ICATIECE), pp. 201- 204, 2019 Tu, Peter, "Artificial Intelligence Community and Self." AI Magazine, vol. 42, no. 1, pp. 25+, 2021. Mani, Ganesh, "Artificial Intelligence's Grand Challenges: Past, Present, and Future." AI Magazine, vol. 42, no. 1, pp. 61+, spring 2021. Plass, Jan L., et al, "The Impact of Individual, Competitive, and Collaborative Mathematics Game Play on Learning, Performance, and Motivation." Journal of educational psychology 105.4, 2021. Mainetti, R., Sedda, A., Ronchetti, M., Bottini, G., & Borghese, “Duckneglect: Video-games based neglect rehabilitation”, Technology & Health Care, 21(2), 97–111. Mills DJ, Milyavskaya M, Mettler J, Heath NL, Derevensky JL, “How do passion for video games and needs frustration explain time spent gaming?” Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne, “Imitation Learning: A Survey of Learning Methods”, ACM Comput, 50, 2, Article 21, March 2017 Rabin, Matthew, “Incorporating Fairness into Game Theory and Economics.”, The American Economic Review, vol. 83, no. 5, American Economic Association, pp. 1281–302, 1993 45 Rios Insua D, Banks D, Rios J., “Modeling Opponents in Adversarial Risk Analysis.”, Risk Analysis: An International Journal. 2016. Ho, Teck-Hua, and Xuanming Su, “Peer-Induced Fairness in Games.” The American Economic Review, vol. 99, no. 5, American Economic Association, pp. 2022–49, 2009 Boyan Bontchev, Olga Georgieva, “Playing style recognition through an adaptive video game”, Computers in Human Behavior, Volume 82, Pages 136-147, 2018. 46 Appendix A Consent Form WEBER STATE UNIVERSITY - INFORMED CONSENT Effect of Changing AI On Player Satisfaction - IRB STUDY # - AY20-21-111 You are invited to participate in a research study of the effect changing AI has on the video game players satisfaction. You were selected as a possible subject because you expressed interest in playing a video game for research. We ask that you read this form and ask any questions you may have before agreeing to be in the study. The study is being conducted by Drew Ackerman from Weber State University. It is funded by Weber State University. STUDY PURPOSE The purpose of this study is to determine if there is a correlation between a player’s satisfaction and the Artificial Intelligence that the game uses. Specifically, the effect that changing AI has on a player’s satisfaction with the game. NUMBER OF PEOPLE TAKING PART IN THE STUDY: If you agree to participate, you will be one of at least 50 subjects who will be participating in this research. PROCEDURES FOR THE STUDY: If you agree to be in the study, you will do the following things: You will be playing a video game and attempting to get the highest score possible. You will have 1 month to play the game however long you want. You can play the game in several sessions, meaning you can leave and come back to the game and resume playing. You will be playing the game from any location that has an internet connection. RISKS OF TAKING PART IN THE STUDY: For face-to-face research, the risks include the possibility of being infected by the novel coronavirus 2019 (COVID-19) or other communicable diseases. There are no risks to participating in this study. There will be no face-to-face research done. BENEFITS OF TAKING PART IN THE STUDY The top 3 participants by high score will receive a reward via a visa gift card. ALTERNATIVES TO TAKING PART IN THE STUDY: Instead of being in the study, you have these options: You don’t have to participate in the study. Taking part in the study is completely optional. COSTS/ COMPENSATION FOR INJURY There is no cost to participating in the study. In the event of physical injury resulting from your participation in this research, necessary medical treatment will be provided to you and billed as part of your medical expenses. Costs not covered by your health care insurer will be your responsibility. Also, it is your responsibility to determine the extent of your health care coverage. There is no program in place for other monetary compensation for such injuries. However, you are not giving up any legal rights or benefits to which you are otherwise entitled. If you are 48 participating in research which is not conducted at a medical facility, you will be responsible for seeking medical care and for the expenses associated with any care received. CONFIDENTIALITY Efforts will be made to keep your personal information confidential. We cannot guarantee absolute confidentiality. Your personal information may be disclosed if required by law. Your identity will be held in confidence in reports in which the study may be published and databases in which results may be stored. Organizations that may inspect and/or copy your research records for quality assurance and data analysis include groups such as the study investigator and his/her research associates, the Weber State University Institutional Review Board or its de- signees, the study sponsor, and (as allowed by law) state or federal agencies, specif ically the Office for Human Research Protections (OHRP) and the Food and Drug Administration (FDA) [for FDA-regulated research and research involving positron- emission scanning], the National Cancer Institute (NCI) [for research funded or supported by NCI], the National Institutes of Health (NIH) [for research funded or supported by NIH], etc., who may need to access your medical and/or research records. CONTACTS FOR QUESTIONS OR PROBLEMS For questions about the study, contact the researcher Drew Ackerman at 480-365- 8304, or mitcha12@live.com or the researcher’s mentor Brian Rague at 801-626- 7377 For questions about your rights as a research participant or to discuss problems, complaints or concerns about a research study, or to obtain information, or offer input, contact the Chair of the IRB Committee IRB@weber.edu. 49 VOLUNTARY NATURE OF STUDY Taking part in this study is voluntary. You may choose not to take part or may leave the study at any time. Leaving the study will not result in any penalty or loss of benefits to which you are entitled. Your decision whether or not to participate in this study will not affect your current or future relations with Weber State University. SUBJECT’S CONSENT In consideration of all of the above, I give my consent to participate in this research study. I will be given a copy of this informed consent document to keep for my records. I agree to take part in this study. 50 Appendix B “The Appointment in Samarra” (as retold by W. Somerset Maugham [1933]) The speaker is Death There was a merchant in Bagdad who sent his servant to market to buy provisions and in a little while the servant came back, white and trembling, and said, Master, just now when I was in the marketplace I was jostled by a woman in the crowd and when I turned I saw it was Death that jostled me. She looked at me and made a threatening gesture, now, lend me your horse, and I will ride away from this city and avoid my fate. I will go to Samarra and there Death will not find me. The merchant lent him his horse, and the servant mounted it, and he dug his spurs in its flanks and as fast as the horse could gallop he went. Then the merchant went down to the marketplace and he saw me standing in the crowd and he came to me and said, Why did you make a threatening gesture to my servant when you saw him this morning? That was not a threatening gesture, I said, it was only a start of surprise. I was astonished to see him in Bagdad, for I had an appointment with him tonight in Samarra. 51