Dude, the Source of Lags Is on Your Computer

De-Yu Chen, Hao-Tsung Yang, and Kuan-Ta Chen
Institute of Information Science, Academia Sinica
derek0208@iis.sinica.edu.tw, htyang@iis.sinica.edu.tw, ktchen@iis.sinica.edu.tw

PDF Version | Contact Us

Abstract

Lag denotes the event when an application fails to respond user inputs in a timely fashion and is considered one of the most common annoyances that impair online gaming experience. Despite years of effort devoted by game developers and network designers trying to overcome lags, gamers still suffer from this annoying phenomenon. It seems to many gamers that lag is an unavoidable part of online gaming and sometimes they just give up fighting it.
In this paper, we tackle the lag problem by investigating the root cause of lags for gamers. We develop a software called Game Experience Monitor (GEM), which monitors the performance of gamers' computers and the quality of network paths during game play, and use the collected traces to correlate players' perceived experience to find out the common cause of lags. Our analysis reveals that, surprisingly, while it is a common belief that the instability of Internet paths is the major cause of lags, the overloading of players' computers in fact plays a more decisive role to lag. It is hoped that this counter-common-belief finding will motivate further research for providing a better infrastructure for gaming and other real-time interactive applications.

1  Introduction

When a gamer is exploring an archaic dungeon or participating in an intense battle in a virtual fantasy world provided by an online game, he must be terrified at the occurrence of "lags," which may largely disrupt the immersive gaming experience. Lag denotes the event when an application fails to respond user inputs in a timely fashion and is considered one of the most common annoyances that impair online gaming experience. Despite years of effort devoted by game developers and network designers trying to overcome lags, gamers still suffer from this annoying phenomenon. It seems to many gamers that lag is an unavoidable part of online gaming and sometimes they just give up fighting it [1].
Lag is a troublesome issue to online games for many reasons. First, there are virtually unlimited factors that may be responsible for lags. In addition to the network delay variations [2] in the Internet, overloading of game servers, access bandwidth competition from neighbors, slow rendering of game scenes due to transient workloads on users' PCs, among other possibilities, can each cause a certain form of lags. Therefore, it is very difficult, if not impossible, for players to identify the root cause of lags when they encounter such annoyances. Players may try any combination of possible solutions found on the Internet, blame game companies, or learn to live with lags. In our previous work [1], we conducted a questionnaire survey regarding three questions: 1) How do players perceive lag? 2) What do players think of the causes of lag? 3) How do players react to lag? Based on the responses from 229 subjects, we confirmed that players mostly struggle with lag because they are unable to identify its root cause and that there is a strong demand for an automatic diagnostic tool that can identify the root cause of lag for gamers.
In this paper, we further tackle the lag problem by investigating the root cause of lags for gamers. We develop a software called Game Experience Monitor (GEM), which monitors the performance of gamers' computers and the quality of network paths between gamers' computers and the game servers during game play, and use the collected traces to correlate players' perceived experience to find out the common cause of lags during online game play. By collaborating with Gamania Inc.1, a top game company in Taiwan, we collect 5,482 minutes of performance traces from 514 players as well as the players' lag experience reports. Our analysis reveals that, surprisingly, while it is a common belief that the instability of Internet paths is the major cause of lags, the overloading of players' computers (either too much workload in processor, too little available memory, or GPU is not powerful enough) in fact plays a more decisive role to the occurrences of lag. We consider our contributions in this paper 2-fold:
  1. We present a general monitoring platform that resides on players' computers and can be used to empirically study the relationship between users' gaming experience and system and network performances.
  2. Based on a set of GEM traces, we find that players' computers, rather than the commonly blamed Internet unstability, tend to be the major source of lags.
It is hoped that this counter-common-belief finding would motivate further research for providing a better infrastructure for gaming and other real-time interactive applications.
The remainder of this paper is organized as follows. Section II describes related works in the area of evaluating user perceptions of network and system QoS. In Section III, we recap a previous work of ours which motivates this study. Section IV introduces our Game Experience monitor. In Section V, we describes how we collaborate with a game company for the data collection. A brief summary of the collected data is also provided. Section VI provided the root cause analysis of lag. Finally, we offer our conclusions of this paper and outline our future work in Section VII.

2  Related Works

A number of works have focused on game players' perceived quality of online games. In [3], Quax et al. conducted an experiment where 12 gamers played an FPS game with each other. A network emulator was used to simulate delay and jitter on the network connection for a subset of gamers. They concluded that compared to other players, the network impairment had a negative influence on the affected players' perceived game quality. They also found that the players could not tolerate delay higher than 60 ms. Zander et al. conducted a similar experiment in [4]. They concluded that delay has a larger impact on the players' perceived quality than packet loss. Meanwhile, they also found that a player's perceived quality is not a sole factor that influence his decision of whether to leave the server immediately or not.
In [5], Wattimena et al. introduced delay, jitter, and packet loss simultaneously on different levels and compared their impacts on gamers' perceived quality. They also came up a regression model that can predict the gamers' perceived quality by the network conditions. In [6], Claypool et al. inquired game players about their perceived game playability and picture quality when the frame rate and frame resolution were controlled. The players' in-game performance were also measured. The authors concluded that in terms of performance, frame rate has a larger impact than frame resolution, while in terms of perceived quality, they are equally important.
This paper differs from previous works from two aspects. First, while the other studies focused on either network conditions or presentation factors (such as resolution and frame rate), we evaluate their effects on gamers' perceived quality of online games. Second, this work is unique in that it is based on an empirical large-scale user traces, rather than on-site controlled experiments; therefore both the system performance metrics and users' self-reported perceived quality are more realistic and the research implications would be more helpful to field applications.

3  Motivating Study

In our previous study [1], we conducted an Internet survey which was motivated to investigate the following questions:
  1. How do players perceive lag?
  2. What do players think of the causes of lag?
  3. How do players react to lag?
We had received a total of 229 complete responses from game players. Here we briefly recap the important implications from the survey results. Readers are referred to [1] for details.

3.1  Perceptions of Lag

As shown in Figure 1(a), a quarter of players claim that they encounter lag frequently (22.7%), while half of players encounter lag occasionally (50.7%). Figure 1(b) shows that only 37% players regard lag as slight, with most players considering lag as serious (21%) and moderate (41.9%). Furthermore, most lag incidents (Figure 1(c)) occur intermittently (34.2%) and lasts for a few seconds (34.5%).
Respondents were also asked to identify the most important factors leading to lag problems. Figure 1(g) show that most players attribute lag to the access link bandwidth of game servers (21.2%) and server equipment (21%). Some players also consider their own PC (19.2%) and the access link bandwidth of their PC (16.9%) as the cause of lag. Moreover, the results show that most players feel that lag is only moderately (34.5%) or weakly (32.8%) related to the time of game play, and most players feel that lag is strongly (34.5%) related to the number of avatars on the screen.
In sum, we find out that 1) players are highly affected by intermittent lag, 2) players usually blame the access link bandwidth of game servers and server equipment for lag, and 3) players believe that lag is strongly related to their online game play time and the number of avatars on the screen.
q13.png(a) Frequency of lag
q14.png(b) Significance of lag
q15.png(c) Duration of lag
q22.png(d) Reactions to lag
q26.png(e) To what degree does lag affect players' game play?
q28.png(f) Where or to whom do players usually complain about their lag problems?
q16.png(g) Players' intuition about root causes of lag
q29.png(h) Strategies to solve lag
Figure 1: Respondents' perceptions of lag

3.2  Reactions to Lag

As Figure 1(d) shows, when players encounter lag, most of them either suffer from lag (64.6%) or ignore lag (17.9%). Figure 1(e) illustrates that even though some players think that lag weakly influences their game play (34.1%), more players claim that lag has a moderate (27.1%) and strong (17.9%) influence on their game play. When players are asked about the relationship between lag and a game they decided to quit, 57.6% of players say that there is a moderate or strong relationship.
When questions focus on whether players have reported lag to the relevant parties, Figure 1(f) shows that most players choose to keep silent (34.9%) or complain about their problems on Internet forums (31.5%). Only a few players contact their ISP (15.8%) or the game company (16.4%).
From the above results we conclude that 1) players usually tolerate or ignore lag, 2) players think that lag influences their game play, 3) players blame lag as the reason they decide to quit online games, and 4) players tend to keep silent or complain on Internet forums when they encounter lag.

3.3  Solutions to Lag

As shown in Figure 1(h), when the respondents are asked which methods they have adopted to cope with lag, a plurality of respondents (23.6%) never adopt any solution. Among the rest of them, 23.6% choose to increase their network bandwidth and 23% choose to upgrade their PC when they encounter lag. Other results shows that only 21.9% of players use network tools to detect lag.
To understand what kind of tools player demand to fight lag, they are asked whether they require root cause diagnostic software or lag mitigation software. The results show that players would like to download and use software that can either diagnose the root cause of lag (86%) or mitigate lag (93.9%).
Overall, our survey identified two points: 1) Most players have no technical background in using network tools to identify the root causes of lag, 2) players demand software tools that can either diagnose the root cause of lag or mitigate lag.

4  Game Experience Monitor

We believe that an ideal way to obtain a better understanding about lag is to directly look into the players' computers and network and also inquire players' perceived quality of game whenever lag may occur. Therefore, we develop a software system which we call the Game Experience Monitor (GEM) for the data collection purpose. It keeps tracks of the performance and activities of a gamer's computer and the quality of network path between the gamer's computer and the game servers during game play. It also provides a user interface for players to report their perceived gaming experience against lag.
gem.png
Figure 2: How Game Experience Monitor works.
As Figure 2 illustrates, GEM is designed to be a software which runs on a player's computer and is invoked only when an online game is launched. GEM monitors the resources available in the operating system, the performance of the game client process, and the quality of network path toward the game server. To achieve this goal, GEM hooks into the game client process using the Windows hook mechanism2 so that it can directly access the kernel variables that were only internally available inside the game client process. We summarize the metrics GEM will collect during game play in Table I and elaborate them below:
Local observations. GEM monitors a wide spectrum of activities and performance metrics on a gamer's local computer:
  1. Processor utilization: Processor is one of the most important components to games. Generally, a more powerful processor enables games to run more smoothly; however, if the processor is busy running other tasks, game performance could drop significantly. Thus, we record the number of CPU cores on the computer along with the processor usage over time, such as processor busy time, number of interrupts it handles per second, etc.
  2. Memory utilization: Memory is another crucial component to gaming performance, as games often place a large amount of data in memory for fast accesses. Thus, we keep tracks of memory utilization factors such as the available memory size and the number of memory accesses during game play.
  3. Game client performance: GEM monitors two performance metrics that we consider critical to players' gaming experience.
    1. Inter-frame time (ift): Inter-frame time accounts for the time period between two consecutive frames (game screens). Since games will strive to refresh its screen in a regular rate (e.g., 50 Hz or higher), a long or unstable inter-frame time would be a strong indicator that the system is overloaded.
    2. Incoming packet stall time (ipst): The game client needs to communicate with the game server frequently, and thus how efficiently the client handles network packets is crucial to gaming performance. When an incoming packet arrives at the gamer's computer, it will be received by the network interface card driver and then wait in the queue of the TCP/IP implementation until the game client calls the recv() socket function to retrieve it. However, in case that the game client process is not active when the packet arrives, it may not be able to resume its execution immediately if the processor is busy handling more urgent interrupts or other processes and cause some extra delay before the packet will be eventually retrieved and handled by the game client. We call such delay time the incoming packet stall time (ipst). The magnitude of incoming packet stall time would also be a representative index of whether the player's computer is overloaded for any reason.
Network observations. Unquestionably the performance of an online game would be largely affected by the quality of network path between a gamer's computer and the game servers. Most people regard Internet conditions as the most likely cause of lag for online gaming; however, we consider that LAN connection quality (especially that of wireless LANs) might also cause serious performance issues. Thus, GEM monitors the quality of LAN and WAN (Internet) path separately. For Internet path quality, we simply infer the round-trip times of packets based on the timestamps and sequence numbers in TCP packets. For LAN quality, we send out a ping packet with the TTL (time-to-live) set to 1 (hop) every 30 seconds and record the round-trip time.
Table 1: Metrics recorded by the game experience monitor client
Category Metric Frequency Description
n_cores Once when the program starts Number of processor cores on the computer
n_processes 1/30 Hz Number of current processes in the system
processor_time 1 Hz Percentage of time the processors are busy
privileged_time 1 Hz Percentage of time the processors spend on privileged instructions
interrupt_time 1 Hz Percentage of time the processors spend on interrupt handling
dpc_time 1 Hz Percentage of time the processors spend on servicing dpc requests
rate_interrupts 1 Hz Number of interrupts the processors receive per second
rate_page_faults 1 Hz Number of page faults per second
available_bytes 1 Hz Current available memory in bytes
rate_page_reads 1 Hz Number of page reads per second
rate_page_writes 1 Hz Number of page writes per second
rate_pages_input 1 Hz Number of pages read from disk per second
rate_pages_output 1 Hz Number of pages written to disk per second
inter_frame_time (ift) Depends on the frame rate Inter-frame time (in ms) between each pair of consecutive frames
incoming_packet_ Time period (in ms) between each packet's arrival at the network
stall_time (ipst) interface and the time the game process receives it
WAN rtt Depends on the packet transmission rate RTT (in ms) of each packet sent to the game server
LAN lan_rtt 1/30 Hz RTT (in ms) to the first hop along the route to the game server

5  Data Collection

We collaborated with Gamania Inc., a top game company in Taiwan, to put GEM system into real use. The game company released a third-person shooting online game called Bubble Fighter in 2012 and, during the beta test period, all the beta test gamers were invited to a "computer and network diagnosis program" based on the GEM system. If a gamer agreed to participate in the program, we instructed him to follow the procedures below:
  1. Download, install, and launch the GEM client.
  2. Run the game client of Bubble Fighter and start to play the game for 10 minutes as usual. The GEM client will monitor all the system resources and network quality during the game session.
  3. Once the 10-minute game session is finished, the participant is asked to report his gaming experience.
  4. The participant's assessment and the GEM traces are uploaded to the GEM server for offline analysis.
Table 2: A summary of collected GEM traces
Period 2012/12/08-2012/12/10
# participants 514
Uploaded compressed data 400+ MB
Monitored gaming session 5,482 minutes
Recorded network packets 12,615,819 (38.36 pkts/sec)
Intercepted send calls 2,142,193 (6.51/sec)
Intercepted recv calls 2,720,961 (8.27/sec)
Intercepted IDirect3DDevice9::Present calls 15,101,096 (45.91/sec)
Intercepted mouse events 1,963,878 (5.97/sec)
Intercepted keyboard events 1,085,800 (3.30/sec)
In order to encourage participation, gamers who complete the test and successfully upload data to our data server receive a number of game credits from the game company. During the beta test from 2012/12/08 to 2012/12/10, 514 gamers have completed the test and uploaded over 400 MB of compressed data to our server. A total of 5,482 minutes of gaming session has been monitored, during which 12,615,819 network packets (average 38.36 packets per second) have been recorded. We have intercepted 2,142,193 send calls (6.51/second), 2,720,961 recv calls (8.27/second), 15,101,096 IDirect3DDevice9::Present calls (45.91/second), 1,963,878 mouse events (5.97/second), and 1,085,800 keyboard events (3.30/second). Table II summarizes the data we have collected.
survey_results_bubble.png
Figure 3: The survey results

5.1  Lag Incident Report

We ask three simple questions regarding the participants' gaming experience during game play:
  1. Did you perceive lag?
  2. How often did you perceive lag?
  3. How did the level of lag change through time?
Figure 3 summarizes the 514 respondents' answers. As the figure shows, 40% of the respondents claim that they encountered lag during the 10-minute gaming session, and most of them report the lag happened at most occasionally. 75% of the respondents feel that the levels of lag did not fluctuate throughout the gaming session.

6  Root Cause Analysis of Lag

In this section, we perform a correlational analysis of players' subjective gaming experience and the objective system metrics collected by GEM in order to identify the major cause of lag. Since lag consists of many kinds of delay coming from many places, it is unlikely to find a single dominant cause. To simplify the problem, we first focus on three possible sources:
Since the GEM traces we collect are sequences of observed data over time and as a result are hard to be used directly for the purpose of analysis, we compute basic statistics of each metric, which are referred to as the features of that metric, and use them in our analysis instead. We use the notation M_V to denote the statistical value V of the performance metric M. The statistical values we use as features include mean (mean), maximum (max), minimum (min), median (median), 99 percentile (99), 95 percentile (95), and standard deviation (sd). For example, rtt_95 denotes the 95 percentile of the round-trip times of all packets sent during the monitored gaming session.

6.1  Correlation Analysis

Table 3: Pearson correlation coefficients between performance metrics and lag incidents
mean median 95 99 max sd
Cor 0.26 0.17 0.25 0.29 0.22 0.25
p-value 1e-4 1e-4 1e-4 1e-4 1e-4 1e-4
Cor 0.06 0.05 0.05 0.02 0.00 0.03
p-value 0.16 0.28 0.26 0.65 0.91 0.45
Cor 0.02 0.02 0.01 0.04 0.05 0.02
p-value 0.65 0.69 0.81 0.34 0.30 0.70
rpart_bubble_ift.png(a) Classification tree of ift
rpart_bubble_rtt.png(b) Classification tree of rtt
rpart_bubble_lan_rtt.png(c) Classification tree of lan_rtt
Figure 4: The classification trees of ift, rtt and lan_rtt
If a metric has an significant effect on lag, it is supposed to be highly correlated with gamers' perception of lag. We use Pearson correlation coefficient as our measurement to compare the effects of ift, rtt and lan_rtt on lag. Table III shows the results. The features of ift have correlation coefficients with gamers' perception of lag between 0.17 to 0.29, with ift_99 and ift_mean reach the highest values of 0.29 and 0.26. The p-values are all smaller than 0.0001, which indicates that the results are all significant. The correlation coefficients between the gamers' perception of lag and the features of rtt and lan_rtt are all below 0.1, which indicates that there are no clear correlations. However, the high p-values (none of them are lower than 0.1) also suggest that the results do not have significant meanings. The fact that inter frame time has a higher correlation with the gamers' perception of lag indicates that overloading of gamers' computers is more likely to be a root cause of lag, which is a surprising result since the poor quality of network paths is blamed as the most important factor that impair online gaming experience for years.

6.2  Classification and Regression Trees Modeling

We believe that if a performance metric is a strong cause of lag, it should be able to be used to predict whether a gamer will perceived lag or not. Therefore, we use the features of the metrics as predictors to build a Classification and regression trees (CART) model [7] for each metric using rpart package3 in R and compare the prediction accuracies of the resulting classification trees, which are depicted in Figure 4.
For the prediction accuracies, we do a 10-fold cross-validation for each tree, in which we randomly choose 10% of the total sample as validation data for 10 times, and computes the average of the rates at which the validation data are correctly classified. The cross-validated accuracies for the classification trees of ift, rtt, and lan_rtt are 67.51%, 56.81%, and 59.34%, respectively. The results show that ift outperforms the other two metrics in terms of prediction accuracy, which echoes the results of the correlation analysis in Section  VI-A. The consistent results of the two different analyses make us more sure of our conclusion that overloading of gamers' computers is actually the major cause of lag.

7  Conclusion and Future Work

In this paper, we tackle a long-standing question for gamers and game developers: What is the major cause of lag? To this aim, we developed a software which is capable of monitoring both the performance of a gamer's computer and the quality of network path during game play. By collaborating with a top game company in Taiwan, we have collected 5,482 minutes of performance traces from 514 players as well as their lag experience reports. We analyze the collected traces via two approaches and conclude that the root causes of lags are usually coming from gamers' computers due to overloading and/or insufficient computation power. We hope that this study will balance the treatment of delay optimization researches as network delay seems to be the overly emphasized while the delays occurred at the end users' side are usually overlooked.

Acknowledgments

This work would not have been possible without the support from Gamania Digital Entertainment. The authors are much indebted to our collaborators and supporters at Gamania, Sirion Lee, Chunhan Huang, Ivy Huang, Karl Hsu, and Albert Liu, among others. The authors also wish to thank the anonymous referees for their constructive criticisms. This work was supported in part by the National Science Council under the grant NSC100-2628-E-001-002-MY3.

References

[1] P.-H. Tseng, N.-C. Wang, R.-M. Lin, and K.-T. Chen, "On The Battle between Online Gamers and Lags," in Proceedings of IEEE CQR 2011, May 2011.
[2] K.-T. Chen, P. Huang, and C.-L. Lei, "How Sensitive are Online Gamers to Network Quality??" Communications of the ACM, vol. 49, no. 11, pp. 34-38, Nov 2006.
[3] P. Quax, P. Monsieurs, W. Lamotte, D. D. Vleeschauwer, and N. Degrande, "Objective and subjective evaluation of the influence of small amounts of delay and jitter on a recent first person shooter game," in Proceedings of ACM NetGames'04, 2004, pp. 152-156.
[4] S. Zander and G. Armitage, "Empirically measuring the qos sensitivity of interactive online game players," in Proceedings of the Australian Telecommunications Networks and Applications Conference 2004.    ATNAC, 2004, pp. 511-518.
[5] A. F. Wattimena, R. E. Kooij, J. M. van Vugt, and O. K. Ahmed, "Predicting the perceived quality of a first person shooter: the quake iv g-model," in Proceedings of ACM NetGames'06, 2006.
[6] M. Claypool, K. Claypool, and F. Damaa, "The effects of frame rate and resolution on users playing first person shooter games," vol. 6071, 2006, pp. 607 101-607 101-11. [Online]. Available: http://dx.doi.org/10.1117/12.648609
[7] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, "Classification and regression trees. wadsworth & brooks," Monterey, CA, 1984.

Footnotes:

1. http://www.gamania.com/eng/
2. The Windows hooking mechanism is invoked by calling the SetWindowsHookEx function. It is frequently used to inject code into other processes.
3. http://www.statmethods.net/advstats/cart.html


Sheng-Wei Chen (also known as Kuan-Ta Chen)
http://www.iis.sinica.edu.tw/~swc 
Last Update September 19, 2017