Grindstone Analysis & Simulation
Executive Summary

I analyzed a RP event known as the Grindstone; a virtual tournament where participants go through one-on-one matches until a winner is decided. A recommendation sought by the client seeks to reduce the amount of time it takes for a winner to be decided. A model was created with a few iterations and the final recommendation is not to change the current system at this time.

Participants arrive two hours prior to the event start based on a distribution derived from interarrival data. At event start, the registration closes and no more participants enter the system. All participants go through a 20 minute debriefing of the rules before being paired up with other participants. Each pair conducts a one-on-one match with a distribution derived from match time data. As the pair finishes a match, the loser is ejected from the system while the winner receives healing from a medic based on a data-derived distribution. Once healed, the winner goes back to the pairing area and waits for another winner to arrive before repeating the process. At the end of the tournament when only one person is left standing, a participant will be declared the winner.

A model was developed with Simio simulation software and simulated over a 10 hour period, starting from 2 hours before the event start. The base case and each iteration went through 100 replications. The model for the base case accurately portrays the real world system.

After examining possible changes to the system, since the match times, healing times, and interarrival times could not be changed right now, I ran iterations which changed the number of medics and overseers available. While both changes decreased utilization in their respective resources, they did not decrease the amount of time it takes to determine a winner, and thus the recommendation is to keep the current system.

Future work can be done to identify other areas for improvement if the current system’s interarrival times, match times, or healing times undergo a significant change. Additionally, if there are changes to how a pair is decided, the model may be expanded on to reflect that change and simulated again.

-----------

Technical Report

Introduction

This report was prepared to analyze a paired one on one tournament setting, known commonly as the Grindstone--a human-run, eSport event executed in a virtual setting. A recommendation was developed to determine if any significant changes are required in staff requirements as well as general feedback about the event.

Assumption

To make the model for the simulation, several assumptions were made:

1. People only arrive two hours prior until the event start; after two hours, no more may arrive and the event continues until the winner is declared.
   
2. There are 4 overseers and 5 medics in the base case.
   
3. Each overseer can watch up to 10 pairs at a given time, but pairs may freely change between overseers during the tournament if need be.
   
4. Any overseer may observe over any pair that is ready to begin their first or next match.
   
5. Data collected accurately represents the real-life scenario.
   
6. Before the matches begin, there is a 20 minute debriefing of the rules and introductions for all combatants.
   
7. All first round matches start at the same time; the matches after the first round are made as participants finish their previous round and pair up with the next available winner.
   
8. All winners of each round must be tended by a medic before they begin their next match or declared the winner.
   
9. All people present in the system have instantaneous travel time.
   
10. Medics and overseers are volunteers and do not have to be paid.
    
11. The tournament can take up to 10 hours total to complete, so the simulation covers a 10 hour period beginning from 2 hours before the event start.
    

Base Case

Before simulating the model, data was collected in the real world environment on three areas over a 3 hour period: match times, medic times, and arrival times. The full information of the distributions derived from the data may be seen on Figure 1, 2, and 3 in the appendix. Arrival times were taken two hours prior to the event starting; on the stopwatch, each newcomer was recorded as a lap. Since all matches began at the same time the timing starts at zero and a lap triggered each time a match finishes. Medic times were recorded in whole minutes, from the time a medic tends to a participant to the time they finish their recovery. Once the values were recorded, they were put into Input Analyzer and assigned a distribution based on best fit. The exception to best fit is interarrival times; lognormal had the lowest square error, but did not generate enough arrivals in the system to mirror the real world system. Beta, while it has a higher square value, generated arrivals similar to the real world system and was used for interarrivals. Match times had a Beta distribution and Medic times had a Gamma distribution for best fit.

The first part of the model is a segregated model away from the main model that restricts newcomers after two hours. A timer element, Start, a source, TimerSource, and a model entity, Timer, are the first pieces created. The source generates Timer for the entity type and has an On Event arrival mode. Start has an offset of 2 hours with a time interval of infinity; the triggering event is Start.Event on TimerSource; once 2 hours passed, the entity is generated and enters the sink TournamentStart.

With the segregated model complete, the main model is next. Two model entities, Participant and Dummy, and two resources, OverseerCapacity and Medic, are made. The initial capacity for OverseerCapacity is 40; as there are four overseers who can watch up to any 10 pairs, 40 slots are available for pairs to seize as they begin their match. A capacity of 40 fits in with assumption 3, as each overseer can watch over 10 pairs at a time, but it doesn’t matter which overseer receives the next pair for a match. Medic’s capacity is 5, as that’s the amount of medics present in the base case.

The first source, ParticipantArrival, generates the Participant entity at an interarrival rate of “17 * Random.Beta(0.567, 3.84)” minutes, taken from the interarrival time data. As demonstrated in the segregated model, under the stopping conditions, the stop event name is “[email protected];” as the Timer entity reaches the TournamentStart sink, the arrivals stop. Additionally, before exiting the source, entities get a state assignment of ParticipantCount with a value of ParticipantCount+1 which counts the number of participants who entered the system.

Another source, PairDummy, generates the Dummy entity with an On Event arrival mode. Like the ParticipantArrival source, it’s also affected by Timer entering TournamentStart. Instead of stopping arrivals, however, it generates one entity and sends it out through the system.

At the combiner, ParticipantHold, Participant enters the member input while Dummy enters the parent input. The combiner’s batch quantity is set to ParticipantCount; all of the participants will be put together into one giant batch, before they exit the combiner and go to the server Debriefing. At Debriefing, the combined entity undergoes a 20 minute process time, per one of the assumptions, before it’s sent to the next combiner Setup.

In Setup, the batch is split apart; the parent output creates the Dummy entity once more and goes into SinkDummy, discarding it for the rest of the simulation, while the member output creates the Participant entities once more. Before the members exit, they’re assigned a boolean state variable Winner with the value false; this variable will determine when someone has won the tournament. The entities pass through a basic node, then enter the next combiner MatchSetup. Participants may enter either the parent or member input; a connector weight of “MatchSetup.ParentInputBuffer.Contents.NumberWaiting == 0” moves the entity to the parent input if none are currently there while a connector weight of “MatchSetup.ParentInputBuffer.Contents.NumberWaiting >=1” moves the entity to the member input if a Participant is already in the parent. As the matches are a one on one competition, the batch quantity of MatchSetup is kept at 1.

As the combined pairs leave MatchSetup, they reach the server, Match, to begin their match. The initial capacity of Match is set to Infinity and the processing time is “10 + 46 * Random.Beta(1.02, 2.47)” minutes, per the distribution found from the match time data. Before a match is processed, a resource from OverseerCapacity is seized. As the match completes processing, the resource is released. The pair goes to another combiner, MatchEnd, where they are split in half; the ‘loser’ of the match exits through the parent output and goes into the sink LoserSink.

After the winner is split off at MatchEnd’s member output, they go to the Recovery server. Initial capacity is set to infinity and the process time is “0.5 + Random.Gamma(1.42, 1.89);” the distribution derived from healing time data. Before the recovery begins, the entity seizes a Medic resource before processing. After processing, the entity releases the Medic resource and travels along the connector.

At the basic node before MatchSetup, MatchEnd, the entity goes through the add-on process trigger “Winning.” At this node the process trigger checks for the condition “Participant.Population.NumberInSystem == 1” If the entity is the only participant left standing, they move onto the assign step; if not, they continue through their matches. At the assign step, the state variable Winner is set to True and moves onto a set node step which changes the entity’s destination to “Input@TournamentWinner” The entity travels the connectors towards the sink TournamentWinner by following the connector to the parent node of MatchSetup then the connector to the sink.

The entity travels on the connector back to the basic node between the combiners Setup and MatchSetup. With the connector forming a loop like this, there is no need to create extra objects in the model. The new winners go through the same parent and member input conditions in MatchSetup like the participants did in the first round and begin their next matches, going through the processes the first round participants did.

When there is only one participant left standing, they activate the add-on process trigger “Winning” at the basic node before MatchSetup and set Winner’s value to True. The entity may now travel on the connector with a weight of “Winner == True” connected to MatchSetup’s parent input and enter the TournamentWinner sink, effectively ending the tournament.

Analysis

The current setup accurately conveys the real world system with the assumptions made in the model. As there are no current options for improving the process times of the servers Debriefing, Match, and Recovery, the remaining solutions, given the constraints, are changing the number of overseers and medics present. For the analysis of the base case and iterations, 100 replications were ran and the results, such as utilization rate and flow time, are pulled from Simio. For statistical analysis, values are compared in MiniTab and give important information, such as p-value, which determines if the change, if any, is significant. The ultimate goal of the analysis is to find what changes decrease the tournament duration, followed by utilization of resources next.

Before further analysis was made on each iteration, a table of the data points was created below with the average values:



Base is the base case, OSUp is iteration 1, and MEDUp is iteration 2. The utilization of the Overseer and medic over a 10 hour period in addition to the flow time under Duration, which indicates a ‘winner’ when a Participant entity enters the TournamentWinner sink. The Flow Time is the average time it takes to determine a winner from 2 hours before start (i.e. 20 EST - 22 EST and beyond).

Base Case

In the base case, the amount of time it took for a tournament to conclude, on average, was 5 hours. The utilizations, however, are hovering around 5% for medics and overseers; this is attributed to assumption 11, where we run the simulation over a period of 10 hours. Since there was a 2 hour waiting period for arrivals, the average event time from start to finish becomes about 3 hours. Despite the small utilization rates, changes between iterations are noticeable; the average data value from Util Overseer % between the Base and OSUp appear different.

For the iterations, two-sample t-tests, where the difference is mu1 - mu2 = 0, compare the data points between the base case and the iteration at a confidence level of 95%; alpha = .05. If p is greater than alpha, then we fail to reject and assume no difference between the data. Three tests are run on the base case and iterations: overseer utilization, medic utilization, and flow time (duration). The full results of the t-tests are in the appendix.

Iteration 1: Medic Count Increase

The initial capacity of the Medic resource goes from 5 to 6.

On overseer utilization, the base had a mean of 5.392 and MEDUp had a mean of 5.416. The p-value is 0.850 at alpha of .05. Since the p-value is significantly larger than the alpha and greater than .1, we fail to reject any difference between the two means and concluded increasing the medic count does not decrease overseer utilization.

For medic utilization, the base case had a mean of 5.76 while MEDUp had a mean of 4.840. The p-value is at 0.000 with alpha of .05, therefore we can reject the null hypothesis and assume adding more medics decreases the medic utilization rate.

For flow time, or duration, the base had a mean of 4.915 and MEDUp had a mean of 4.891. P-value is at 0.755 with an alpha of .05, so we fail to reject and do not find a difference in flow time between the two data sets.

Though there is a significant decrease in medic utilization by adding an additional medic, the overall flow time is barely negligible; even with adding an extra resource, the entities will still take the same amount of time to pass through the system.

Iteration 2: Overseer Count Increase

The initial capacity of OverseerCapacity changes from 40 to 50; one additional overseer is brought on board and can view 10 people at a time like the others, so 50 slots are now open for matches.

For overseer utilization, the base had a mean of 5.392 and OSUp had a mean of 4.313. The p-value is 0.000 at alpha of .05, therefore assuming there is a difference in overseer utilization between the base and the iteration and adding more overseers reduces overseer utilization.

For medic utilization, both the base and OSUP had a mean of 5.76. The p-value is 1.000 at an alpha of .05, so we may safely assume there is no significant improvement to medic utilization by adding additional overseers.

On the duration, or flow time, the means were the same again with the base and OSUP at 4.915. The p-value is at 1.000 with an alpha of .05, so we may assume there is also no significant improvement in flow time and getting participants through the system quickly by adding more overseers.

Like Iteration 1, increasing the amount of the respect resource decreases the utilization rate of that resource. Despite the utilization decrease, the parts going through the system will still take the same amount of time.

Conclusion

The model produced is an accurate representation of the real world system. Further increasing the number of medics or overseers on hand decrease only their respective utilization rates and do not decrease the amount of time it takes for parts to go through the system. Since no other improvements may be made without significantly changing the interarrival times, match times, or healing times data, the company should not change the current system at this time.

Future work can be done on the model in a variety of ways. Changing the interarrival rates, overseer capacities, match times, healing times, and pairing methods may change the number of overseers and medics required for the event. Should there be a significant change in any of the four items mentioned earlier, the model can be adjusted and tested once more.

Addendum

I realize there are plenty of things not done in the simulation here that are entirely reflective of the true GS, such as travel time between combatants and the fact that matches are independent of overseers. Ah well. In the grand scheme of things, I thought those two items were negligible.