How Sensitive are Online Gamers to Network Quality?
Kuan-Ta Chen,
Polly Huang,
and Chin-Laung Lei (To appear in Communications of the ACM, Special Issue on Entertainment Networking - Recreational Use of IP Networks, November, 2006.)
Network quality is generally considered critical to real-time,
interactive, online game playing. Yet, even though today's
best-effort Internet does not provide quality-of-service (QoS)
guarantees, online gaming have been becoming increasingly
popular. This may be because QoS is not really a "must-have,"
or because players are simply accustomed to struggling with
unfavorable network conditions.
Ironically, despite the growing popularity of online games,
complaints about high "ping-times" or "lags"
continue to surge
in game player forums5.
It would appear that most players see network latency and loss as
major hindrances to enjoyable gaming experiences. This article
investigates whether players are really sensitive to
network quality as they claim. If so, we would like to answer:
"Do players quit games earlier because of unsatisfactory
network conditions?"
A number of studies have evaluated the effect of network quality
on online gamers [HB03,[OH03,[Arm03,[NC04,[BCL+04,[ZA04,[YII05,[CHW+06]. In
most studies, a series of games is played in a controlled network
environment, where the gaming experience of subjects is graded
either subjectively or objectively. Subjective tests require
participants to report their feelings about playing games with
different levels of network QoS. While this approach might
capture human perceptions effectively in a controlled situation,
it cannot measure how players react to poor game quality
in real life, because other factors may outweigh gamers'
QoS-intolerance and affect their decisions. Besides, it is very
costly for such studies to scale up by including more human
subjects. Meanwhile, objective evaluations are usually based on
user performance in a specific context, such as the number of
kills in shooting games, the time taken to complete each lap in
racing games, or the capital accumulated in strategy games.
However, game scores are highly dependent on player skills, game
design, and content, so the results are not comparable and
generalizable across different games.
From a psychological perspective, the very pleasing sensation
that players experience in online games is analogous to being in
the flow state after taking mood-changing substances like
marijuana. A study by Said et al. found that
the most significant factors related to the flow experience in
online gaming are skill, challenge, and
involvement[IML03]. The game playing experience
can be described as a pleasurable and exciting activity that
makes gamers unaware of the time that passes while they stay in
the game.
According to the theory, if the feeling of involvement in the
virtual world is diminished by network lags, users will be more
conscious of the real world and their sense of time
distortion will be mitigated. Furthermore, players may simply
decide to quit a game as soon as they detect unacceptable lags.
Therefore, we conjecture that game playing time is
affected, to some extent, by the network quality that users
experience.
There are certainly exogenous reasons other than the quality of
network conditions that could affect gamers' decisions to
continue with a game or leave it. For example, players may leave
a game due to their schedule constraints or mental condition. On
the other hand, they may be tied by social bonds, no matter how
unstable the game play is. Although the behavior of individual
players is highly variable and unpredictable, we show that,
overall, there is a consistent and strong link between
players' departure decision and network QoS.
Network quality vs. game playing time
Our conjecture is validated by real-life traces taken from a
commercial MMORPG, ShenZhou Online[UT].
MMORPGs (Massively Multiplayer Online Role-Playing Games), such
as EverQuest and World of Warcraft,
are networked computer role-playing games in which a large number of
players interact with one another in a virtual world. While many
genres of games are round- or stage-based, a design that allows
players to regain consciousness of the real world, the adventure in
role-playing games is continuous and there is no explicit
mechanism that forces players to take a break. The uninterrupted
sense of immersion in a virtual environment is considered one of
the main attractions of MMORPGs because it entices players into a
flow experience; the other major attractions are the elaborate
reward cycles and the social networks [Yee].
This implies that network quality, a key factor in maintaining a
perfect sense of immersion, strongly influences the flow
experience of MMORPG players; therefore, the game playing time
should be shorter if network conditions that users experience is
unsatisfactory.
Another reason for investigating MMORPGs is that they are
relatively slow-paced compared to other popular genres, such as
first-person shooting games, that usually require players to make
split-second decisions. In addition to the pace of a game, there
is a great deal of difference in how players control their
virtual characters. In fast-action genres, such as shooting
games, players must instruct characters "what" actions to take
and "how" to perform those actions; for example, to move a
character to a new location, a player must control each step it
takes (such as three steps west followed by five steps north). On
the other hand, in slow-action genres, such as MMORPGs and war
strategy games, players only instruct characters "what" to do,
i.e., they only need to point out the location the character
should move to, and it will automatically head toward the
destination via a route that is either pre-determined by game
designers or computed on-the-fly. By any standard, MMORPGs are
slow-action games that undoubtedly have less stringent service
requirements than fast-action games. Therefore, they could be
seen as a baseline of
real-time interactive online games such that if network QoS
frustrates MMORPG players, it should also affect players of other
game genres.
ShenZhou Online is a commercial, medium-size, fantasy
MMORPG, in which thousands of players are online at any time. To
play, participants must pay a monthly subscription fee at a
convenience store or online. As is typical of MMORPGs, players
can engage in fights with random creatures, train themselves for
particular skills, partake in marketplace commerce, or take on a
quest.
With the help of the ShenZhou Online staff, we monitored
all inbound/outbound game traffic at the server side. A total of
15,140 game sessions were recorded over two days. The network
performance for each session was extracted based on the sequence
number and flags in the TCP packet header, where the network
latency was computed by the difference between the
acknowledgement and delivery time of non-retransmitted game
packets, and the network loss rate was inferred based on the
receipt (or not) of TCP acknowledgements. The observed players
were spread over thirteen countries and hundreds of autonomous
systems, which manifests the heterogeneity of network path
characteristics and therefore the generality of this trace.
On average, players stayed for 100 minutes after joining a
game. This is consistent with the statistics of Korean MMORPGs
played in Japan [4Ga], which show that the average
session time is between 80 and 120 minutes. However, the
differences in individual game playing time are very large; for
example, the shortest 20% of sessions span less than 40
minutes, but the top 20% of players spend more than 8 hours
continuously in a game.
We use graphical plots to illustrate the difference in game
playing time of sessions which experienced different levels of
network quality. Fig. 1 depicts the association of
game playing time with network latency, network delay variation,
and network loss rate, respectively. All three plots indicate
that the more serious the network impairment a player
experienced, the sooner he/she left the game. The changes in
game playing time are surprisingly significant. For instance,
gamers who experienced 150 ms latency played for 4 hours on
average, but those who experienced 250 ms latency only played
for 1 hour on average-a high ratio of 4:1. Moreover,
variations in network delay and network loss induce more
variation in game playing time (note the range of the y-axis of
the graphs).
Now that we have demonstrated that players are not only
sensitive, but also reactive, to network quality;
however, how to define "good quality" or "bad quality"
remains a problem. For example, given two QoS configurations:
"low latency with moderate variation" and "moderate latency
with low variation," which one is better? To answer this
problem, we propose a model that describes the changes in game
playing time due to network QoS. This allows us to grade the
overall quality of game playing based on a set of network
performance metrics, such as latency and loss, in terms of user
satisfaction.
Figure 1: The relationship between game session time and
network QoS factors
Modeling of game playing time
We find that the survival model, which is often used in the
medical field to describe the relationship between patients'
survival time and the treatment they receive, provides a good fit
to describe the reactions of game players to network
quality [CHW+06]. The derived model takes network QoS
factors as the input, and computes the departure rate of online
players as the output. The regression equation is defined as
follows:
departure rate
∝ exp(
1.6×network latency+
9.2×network delay variation+
0.2×log(network loss rate)).
(1)
This equation illustrates that the player departure rate is
roughly proportional to the exponent of the weighted sum of
certain network performance metrics, where the weights reflect
the impact of each type of network impairment. The coefficients
can be interpreted by the ratio of the departure rates of two
sessions. For example, suppose two players, A and B, join a
game at the same time and experience similar levels of network
loss and delay variations, except that their network latency is
100 ms and 200 ms respectively. The ratio of their respective
departure rates can then be computed by exp((0.2 − 0.1)×1.6) ≈ 1.2, where 1.6 is the coefficient of network
latency. That is, at every moment during the game, the
probability that A will leave the game is 1.2 times the
probability that B will leave.
Given the strong relationship between game playing time and
network QoS,
we can "predict" the former if the latter is known. Forecasting
when a player will leave a game could provide useful hints for
system performance optimization and resource allocation. To
assess the feasibility of forecasting game playing time, in
Fig. 2, we compare the actual time and
model-predicted time. On the graph, sessions are sorted and
grouped by their risks, defined according to Eq. 1,
to estimate the level of player intolerance of poor network
conditions. We observe that, at the macro level, the prediction
is rather close to the actual time observed, which suggests that
a service provider can predict how long a player will stay when
he/she joins a game and optimize resource allocations
accordingly.
Figure 2: Actual vs. model-predicted game playing time for
session groups sorted by their risks
Please note that while the methodology we presented can be
readily applied on all kinds of online games, the exact equation
about players' QoS-sensitivity may depend on specific game
design, such as the transport protocol used, and vary between
different games.
Applications of Player Sensitivity to Network QoS
We have shown that unsatisfactory network conditions discourage
users from continuing with a game. Even though we are unable to
change the fact that the Internet today is not QoS-guaranteed,
which may leave players unsatisfied due to poor network
conditions, we can still improve matters by exploiting players'
sensitivity to network QoS.
Improving user satisfaction. Given that we can quantify
the risk of players leaving a game due to unsatisfactory QoS,
systems could be designed to automatically adapt to network
quality in real-time in order to improve user satisfaction. For
example, we could enhance the smoothness of game playing in
high-risk sessions by increasing the packet rate (if the high
risks are caused by long propagation delay or random loss on a
noisy link rather than transient congestion) or the degree of
data redundancy so that the players would be less likely to leave
prematurely. Scarce resources, such as the processing power (for
handling player requests and computing game states), or the
network bandwidth (for dispatching the latest game states), could
be allocated more appropriately to sessions based on their risk
scores. More specifically, based on the risk scores, resource
allocation could be deliberately biased toward high-risk sessions
that experience poor network quality, while simultaneously
maintaining a reasonable level of player satisfaction in low-risk
sessions.
Optimization of network infrastructure. Based on
Eq. 1, network quality can be assessed by a single
value so that different levels of quality can be compared in
terms of user satisfaction. For example, according to our
scoring, players prefer a setting of 200 ms latency and 0.1%
loss rate rather than a setting of 100 ms latency and 1%
loss rate, because packet loss is less tolerable than network
latency. As different performance metrics are usually design
trade-offs, this score can be used to optimize overall user
satisfaction. For instance, as our model indicates that players
are less tolerant of large delay variations than high latency,
providing a smoothing buffer at the client side, which would
incur additional latency but smooth the pace of game playing,
would be a plus as it improves the overall gaming experience.
Network troubleshooting. To provide continuous
high-quality game services,
providers monitor network conditions between servers and customer
networks to detect problems in real-time before customers'
complaints flood the customer service center. However,
considering the large number of online users (hundreds of
thousands is not uncommon for popular games), it would be
prohibitively expensive, even unpractical, to monitor the status
of network paths between all inter-communicated networks in real
time. Instead, we can track users' gaming time, which is much
more cost-effective. Since online gamers are sensitive to network
conditions, a series of unusual (premature) departures within a
short period might indicate abnormal network conditions and
appropriate remedial action could be triggered automatically.
Conclusion
In recent years, QoS provisioning for real-time interactive
applications, such as online gaming, has been actively discussed
because of the popularity of such applications and the complexity
of the problem. One of the main obstacles to provide satisfactory
user experience is that, unlike system-level performance metrics,
such as bandwidth or latency, user satisfaction is
intangible and unmeasurable. The key to this problem is to measure
users' opinions about network performance objectively and
efficiently. We have shown that game playing time is strongly
related to network QoS and is thus a potential indicator of user
satisfaction. Nevertheless, users' perceptions are inevitably
influenced by the design and implementation of an application. How
to generalize the measure of user experience so that the
satisfaction measures for different applications can be normalized
and compared with each other remains an open question.
References
[4Ga]
4Gamer.net.
Gametrics weekly Korea MMORPG population survey.
http://www.4gamer.net/specials/gametrics/gametrics.shtml.[Arm03]
Grenville Armitage.
An experimental estimation of latency sensitivity in multiplayer
Quake 3.
In 11th IEEE International Conference on Networks (ICON), 2003.
[BCL+04]
Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, and
Mark Claypool.
The effects of loss and latency on user performance in Unreal
Tournament 2003.
In Proceedings of NetGames'04, pages 144-151. ACM Press, 2004.
[CHW+06]Kuan-Ta Chen, Polly Huang, Guo-Shiuan Wang, Chun-Ying Huang, and Chin-Laung Lei.
On the Sensitivity of Online Game Playing Time to Network QoS.
In Proceedings of IEEE INFOCOM'06, Barcelona, Spain, April
2006.
[HB03]
Tristan Henderson and Saleem Bhatti.
Networked games: a QoS-sensitive application for
QoS-insensitive users?
In RIPQoS '03: Proceedings of the ACM SIGCOMM Workshop on
Revisiting IP QoS, pages 141-147. ACM Press, 2003.
[IML03]
Said Ila, Dick Mizerski, and Desmond Lam.
Comparing the effect of habit in the online game play of australian
and indonesian gamers.
In Proceedings of the Australia and New Zealand Marketing
Association Conference, 2003.
[NC04]
James Nichols and Mark Claypool.
The effects of latency on online madden NFL football.
In Proceedings of NOSSDAV'04, pages 146-151. ACM Press, 2004.
[OH03]
Manuel Oliveira and Tristan Henderson.
What online gamers really think of the internet?
In Proceedings of NetGames'03, pages 185-193. ACM Press, 2003.
[UT]
UserJoy Technology.
ShenZhou Online.
http://www.ewsoft.com.tw/.[Yee]
Nick Yee.
The Daedalus project.
http://www.nickyee.com/daedalus.[YII05]
Takahiro Yasui, Yutaka Ishibashi, and Tomohito Ikedo.
Influences of network latency and packet loss on consistency in
networked racing games.
In Proceedings of NetGames'05, pages 1-8. ACM Press, 2005.
[ZA04]
Sebastian Zander and Grenville Armitage.
Empirically measuring the QoS sensitivity of interactive online
game players.
In Australian Telecommunications Networks and Applications
Conference (ATNAC), 2004.
Footnotes:
1. This work is supported in part by the National Science Council under
the Grants No. NSC 94-2218-E-002-074 and NSC 95-3114-P-001-001-Y02, and by the Taiwan Information Security
Center (TWISC), National Science Council under the Grants No.
NSC 94-3114-P-001-001Y and NSC 94-3114-P-011-001.
2. Kuan-Ta Chen (ktchen@iis.sinica.edu.tw) is an assistant research fellow in the Institute of Information Science at Academia Sinica, Taipei, Taiwan.
3. Polly Huang (phuang@cc.ee.ntu.edu.tw) is an associate professor in the Department of Electrical Engineering and Director of the Network and Systems Laboratory at National Taiwan University, Taipei, Taiwan.
4. Chin-Laung Lei (lei@cc.ee.ntu.edu.tw) is a professor in the Department of Electrical Engineering and Director of the Distributed Computing and Network Security Laboratory at National Taiwan University, Taipei, Taiwan.
5. http://www.gamedev.net/community/forums/Sheng-Wei Chen (also known as Kuan-Ta Chen) http://www.iis.sinica.edu.tw/~swc
Last Update September 28, 2019