Mobile cloud gaming allows gamers to play games on resource-constrained mobile
devices, and a measurement study to quality the client performance and energy
consumption is crucial to attract and retain the gamers. In this paper, we
adopt an open source cloud gaming platform to conduct extensive experiments on real
mobile clients. Our experiment results show several insights that are of
interests to researchers, developers, and gamers. First, compared to mobile
native games, mobile cloud games save energy by up to 30%. Second, the
hardware video coders achieve higher frame rates but suffer from a small
unnecessary buffering delay, and thus is less ideal for face-pace games. Third,
the frame rate, bit rate, and resolution all affect the decoders' resource
consumption, while frame rate imposes the highest impact. Last, 3G cellular
networks incur 30%-45% more energy consumption than WiFi networks, and the
event processing of touch screens is also energy-hungry. These findings shed
some light on the further enhancements of the emerging mobile cloud gaming platforms.
1 Introduction
Increasingly more companies, such as OnLive, GaiKai, and Ubitus, offer
on-demand cloud games to users. These games run on powerful cloud servers,
and the game scenes are captured, encoded, and streamed to thin clients running
on desktops, laptops, and TV set-top boxes. Each client renders the game scenes
and takes the inputs from a gamer. The inputs are sent back to cloud servers,
in order to allow gamers to interact with games. Cloud gaming has attracted a
lot of attentions [16] and is predicted to be a major growing sector
of the game industry in the next few years [4]. Recently, we start to see
mobile cloud gaming services, for example in the U.S. [6] and
Japan [5], that target the staggering number of mobile device users.
With the wide deployment of 4G cellular networks, mobile cloud gaming enables
gamers to play their favorite games anywhere and anytime.
Mobile devices, such as tablets and smartphones, have limited computation power
and are battery-powered. Therefore, running mobile clients on these
resource-constrained devices may lead to inferior performance and high energy
consumption. For example, the gaming frame rate may become too low for smooth
game play due to insufficient CPU power to execute software video decoders.
This results in degraded gaming quality and may drive the gamers away. On the
other hand, when gamers play cloud games, the communication, computing, and display components on mobile devices all consume nontrivial
energy, which may quickly drain the
battery and prevent gamers from using their mobile devices for other purposes,
such as making phone calls. Hence, carefully measuring the performance and
energy consumption of mobile clients is critical to the success of the new
mobile cloud gaming ecosystem.
In this paper, we adopt an open source cloud gaming
platform [10,11], called GamingAnywhere (GA), to setup a real
mobile cloud gaming testbed. We conduct extensive experiments on the testbed to
answer the following questions.
Does running cloud games save energy compared to native mobile
games? While in first glance, offloading games to cloud servers saves
significant amount of computation energy, doing so however also increases the
communication energy consumption. This is partially due to the nature of
constant video streams incurred by cloud games, which prohibits the wireless
interfaces from being turned off when idling. Hence, a carefully designed
measurement study is required to compare their total client energy consumptions.
What are the pros and cons of the hardware video decoder? The video
decoder represents the majority of CPU workload on the mobile client, which in
turn slows down the overall frame rendering and consumes more energy. One
possible solution is to leverage the hardware video decoders on modern mobile
devices. However, hardware video decoders are usually less configurable than
the software ones. Thus, detailed empirical comparisons may shed light for future
mobile cloud gaming researchers and developers on the tradeoff between the
hardware and software video decoders.
How does the server configuration affect the client performance? The
cloud gaming servers are highly configurable, and selecting the best
configuration itself is a challenging task. Therefore, a comprehensive
set of experiments is required to throughly quantify the impacts of different
server system parameters, such as frame rates and resolutions, on the client
performance and energy consumption. The measurement results are useful to
researcher, developers, and gamers to better configure their cloud gaming
servers.
Does the mobile client suffer from any power-hungry software
components other than video decoders? Although the software video decoders
incur the majority of the CPU workload, which consumes high energy, we also
need to identify other software components, if any, that consume nontrivial
energy. This is because, the battery technology is not improved as fast as
other hardware components [7], and the overall energy consumption is crucial to
retain mobile cloud gamers.
Our extensive experiments and in-depth analysis depict several insights that
lead to design suggestions for future developments of mobile cloud gaming
platforms. To the best of our knowledge, similar measurement studies have not
been rigorously done in the literature.
2 Related Work
Various optimization approaches have been proposed for mobile cloud gaming
platforms, which can be classified into: specialized video
codecs [9,18,3] and system
adaptations [1,19,15], Specialized video codecs leverage the
properties of computer rendered graphics to reduce the downlink bandwidth
consumption of mobile cloud games. In particular, Hemmati et al. [9]
selectively encode game objects to save network bandwidth and rendering power
while maintaining gaming quality. Shu et al. [18] adopt 3D warping for
light-weight post-rendering manipulation on mobile clients, in order to reduce
the network bandwidth and cope with network delay. Chuah and
Cheung [3] render low-quality game scenes on mobile devices while
streaming the difference between low- and full-quality game scenes from cloud
servers, so as to trade client computation complexity for communication
complexity. System adaptations dynamically adjust the system resources across
multiple distributed servers for better overall performance. Cai et
al. [1] divide computer games into small components, and dynamically
move these components across multiple distributed servers to meet the demands
from mobile gamers. Wang and Dey [19] adjust the visual effect levels of
computer games to trade off the server computation load and user-perceived
quality. Liu et al. [15] present a subjective model to approximate the
user experience under diverse video contents, coding parameters, and network
conditions, in order to guide their adaptive rendering component for better
gaming quality. These optimization
techniques [9,18,3,1,19,15] are complementary to
the measurement studies presented in this paper.
Both objective measurement studies [2,17] and subjective user
studies [12,13] on cloud gaming using desktops have been done in
the literature. We focus on mobile cloud gaming, which has only been recently
considered [14,11,8]. Lampe et al. [14] adopt three
performance metrics: latency, energy, and cost, trying to demonstrate the
feasibility of mobile cloud gaming. In our earlier work [11], we
conduct a user study to quantify the impact of different configurations on
mobile gamer satisfaction. In contrast, the current paper concentrates on the
objective measurements on mobile devices. Hans et al. [8] measure
the energy consumption of Android smartphones running a cloud gaming client,
and is probably the closest work to ours. While their findings on energy
consumption are in-line with ours, Hans et al. [8] is different from
the current paper for two main reasons: (i) we measure both client performance
and energy consumption, while they focus on energy, and (ii) we deploy several
actual games in our cloud gaming platform, while their CPU/GPU workloads are
emulated.
The current paper is built upon our earlier work on developing the first open
source cloud gaming platform [10], and porting the cloud gaming client
to mobile devices [11]. Our mobile cloud gaming platform is
transparent to existing PC games, and is of interests to researchers and
developers for experiments and further enhancements.
3 Methodology
3.1 Environment Setup
Figure 1 shows the GA testbed used in our experiments. The
testbed consists of a PC server and a mobile client connected via a
wireless access (a campus WiFi or a 3G cellular network).
We install five games from different genres on the GA server:
Super Smash Bros, Limbo, Batman, Mario Kart, and Zelda. Super Smash Bros is a
fighting game, Limbo is a 2D scrolled adventure game, Batman is a 3D adventure
game, Mario Kart is a 3D racing game, and Zelda is an RPG. We study the GA
mobile client's performance and energy consumption using these five games, and
report the sample results from Super Smash Bros if not otherwise specified.
For comparing cloud and native games, we adopt a cross-platform OpenGL game:
GLTron, which runs on both the PC server and Android devices. GLTron is a 3D
snake-like game.
The PC server has an Intel Q6600 2.4 Ghz quad-core CPU and runs Windows 7. We
consider two mobile devices: an ASUS Nexus 7 tablet and a Sony Xperia Z
smartphone. The tablet has a Tegra 3 1.2 quad-core process with 1 GB ram, and
the smartphone has an S4 1.5 GHz quad-core processor wth 2 GB ram. Both mobile
devices run Android 4.4.2.
We adopt the following tools to collect measurement results.
UseMon is an Android application that reports the CPU
utilization of each CPU core.
Current Widget is an Android application that measures the
current and voltage of the device, which allows us to calculate the energy
consumption. For clarify, we report power consumption instead in some experiments.
During the experiments, we set the screen brightness to medium, and always keep
the battery level above 70% to avoid noises due to battery's nonlinear
discharging characteristics (details are given in
Section III-C).
Figure 1: The GA experiment testbed used throughout this paper.
3.2 Controlled Parameters
Table I lists the controlled parameters during the experiments.
First our mobile client supports both software and hardware video decoders. The
software decoder is provided by the ffmpeg project, and the hardware
decoder is accessed via Android's MediaCodec framework. We use the
popular H.264 coding standard, which is supported by ffmpeg and both mobile
devices' hardware codecs. Second, we selectively disable and enable the
controller on mobile devices, which is a transparent overlay over the video
surface. When the controller is disabled, we play the games on the PC server.
This is to isolate the additional energy consumption due to: (i) activating
touch screens and (ii) handling the user input events. The remaining three
parameters, resolution, bitrate, and frame rate, are for video codecs. In each
experiment, we fix two video codec parameters, and vary the other one. We let
640x480, 30 fps (frame per second), and 3 Mbps be the default settings, if not otherwise
specified. The goal is to quantify the impacts of different parameters on
client performance and energy consumptions.
Table 1: Controlled Parameters
Parameter
Value
Decoder
hardware, software
Controller
disabled, enabled
Video codec parameter‡
Resolution
640x480, 960x720, 1280x720
Bitrate
1Mbps, 3Mbps, 5Mbps
Frame rate
10fps, 30fps, 50fps
‡ Default values are highlighted in boldface.
3.3 Baseline Energy Measurement
We measure the baseline energy consumptions before conducting the
experiments.
We close all irrelevant applications and services, turn on the display,
and set brightness to medium.
We find that the CPU utilization is close to zero. We measure the current and
voltage for each mobile device, sampled at 1 Hz.
The results are shown in Figure 2.
On both devices, we observe that when the battery level reduces,
the voltage gets lower and the current gets higher.
When the battery level is lower than 60%, the current
exceeds the average. For fair comparisons, we only conduct experiments when
battery level is higher than 70%.
Based on the measurements, the baseline power consumption for Nexus 7
and Xperia Z are 1.7 W and 1.1 W, respectively.
Figure 2: The voltage and current levels measured under the baseline configuration.
4 Measurement Results
4.1 Software vs. Hardware Video Decoders
Table II shows the frame rates achieved by the software
decoders. This table reveals that while software decoders work well when the
resolution, frame rate, and bitrate are low, they fail to achieve the
configured frame rate when these video codec parameters are high. This can be
attributed to the limited CPU resources on the mobile devices. We then switch
to the hardware decoders, which run faster but do not report the achieved frame rate. We
observe fairly constant frame rate under different video codec parameters. We
make an interesting observation: the hardware decoders on both mobile devices buffer 1 or 2
decoded frames, which lead to unnecessary delay. Such limitation renders the
hardware decoders less suitable to mobile cloud gaming platforms with longer
network latency and fast-pace games.
Next, we zoom into two video codec configurations: (i) light, with
640x480, 10 fps, and 3 Mbps and (ii) heavy, with 1280x720, 30 fps, and 3
Mbps. We run each experiment for 15 minutes, and plot the CDF curves of
per-core CPU utilization in Figure 3, where N7 and XZ
represent the tablet and smartphone respectively. We first observe that, for
each mobile device, the two curves (light and heavy) of the hardware codecs are
very close. This validates the aforementioned observation: the hardware
decoders achieve the same frame rate under different video codec parameters. In
contrast, for each mobile device, the gap between two curves of the software
codec is much larger. Last, our measurements on the power consumption leads to similar
observation (figure not shown due to the space limitations). The hardware decoders consume power levels that are about two
times of the baseline, which is independent to video codec parameters. In
contrast, the software decoders are sensitive to video codec parameters, and
draw power consumptions that are between two and three times of the baseline.
Table 2: Achieved Decoder Frame Rate (in fps)
Nexus 7
Xperia Z
Configured
10
30
50
10
30
50
1280x720, 5 Mbps
1280x720, 3 Mbps
1280x720, 1 Mbps
960x720, 5 Mbps
960x720, 3 Mbps
960x720, 1 Mbps
640x480, 5 Mbps
640x480, 3 Mbps
640x480, 1 Mbps
Figure 3: Comparison of CPU utilization with the hardware and
software decoders under different loads.
4.2 Video Codec Parameters
Figure 4: Measured CPU utilization and power consumption under various codec parameters.
We next present the CPU utilization and power consumption under different video
codec parameters. We repeat the 3-minute experiment 5 times, collect samples at
1 Hz, and give the average results with minimum and maximum in
Figure 4. This figure reports average CPU utilization,
i.e., 25% CPU utilization is equivalent to a fully-loaded CPU core. We make
two observations: (i) higher bitrates, frame rates, and resolutions consume
more resources and (ii) the software decoders consume more resources. Next, we
take a closer look at how each codec parameter affects the performance of the
hardware decoders. We do not consider the software decoders, because neither of
the considered mobile devices can keep up with the high frame rate. We define
the parameter impact factor as follows.
Given a parameter p and a function fp that quantifies the load
of p based on its parameters.
Suppose p is altered from ci to cj, we write the increased load
Lp as
Lp =
fp(cj)−fp(ci)
fp(ci)
.
(1)
We also measure the battery level differences mi and mj for ci and
cj, respectively.
The increased overhead Op for p is defined as
Op =
mj−mi
mi
.
(2)
Last, the impact factor for parameter p is written as [(Op)/(Lp)].
Note that we measure the battery level difference to define the parameter
impact factor because CPU utilization does not fully reflect system loads, as
some workload is offloaded to the hardware decoders. Table III gives
the parameter impact factors. This table shows that the frame rate has the
highest impact, and the resolution has the lowest.
Table 3: The parameter impact factors for hardware decoders
Nexus 7
Xperia Z
Param. Change†
a→b
b→c
a→c
a→b
b→c
a→c
Bitrate
+0.14
+0.02
+0.13
+0.07
+0.03
+0.07
Frame rate
+0.06
+0.10
+0.10
+0.11
+0.09
+0.14
Resolution
+0.07
-0.17
-0.01
+0.01
-0.03
-0.01
† a, b, and c are the minimal, median,
and maximal values of each parameter.
4.3 Game Genres
We report the CPU utilization and energy consumption of different game genres in
Figure 5. All the games are configured to stream game scenes at
1280x720, although only Windows games (Limbo, Batman, and GLTron) support
1280x720: the game scenes captured from N64 emulator (Super Smash Bros, Mario
Kart, and Zelda) have to be up-sampled to 1280x720 before being streamed.
Therefore, the game scenes from the N64 emulated games contain less details,
which in turn consume less resources. The power consumption fluctuations
caused by all game genres are within ±5%. If we separate the Windows and N64 games,
the fluctuations are about ±2%. We conclude that game
genre has very little impact on cloud gaming CPU utilization and power
consumption.
Figure 5: CPU utilization and power consumption for different game genres.
4.4 Cloud versus Native Games
Next, we play Super Smash Bros and GLTron as cloud games and
GLTron as a native game.
Cloud games are configured to stream at 1280x720.
The results are shown in Figure 6.
In the figure, "Cloud#1" and "Cloud#2" correspond to Super Smash Bros and
GLTron, respectively.
"Native" is the Android version of GLTron.
It is clear that the native game consumes much more resources than cloud games:
the CPU consumption is doubled and the power consumption is also increased by more than
30%. We emphasize that the GLTron game is not very visually-rich, and yet
running it natively incurs nontrivial resource consumption. The resource
consumption gap between the cloud and native games will be even larger for modern
3D games. Last, we make another observation: enabling the controller
results in additional resource consumption. We take a deeper look at this
observation below.
Figure 6: CPU utilization and power consumption for
cloud and native games.
4.5 Other Components
We study the impact of other components on resource consumption,
including the wireless access links and touch screens.
Figure 7 shows the resource consumptions by using
different wireless access links. We only report results from
Xperia Z because Nexus 7 does not have a 3G module.
We work with the default codec configuration (640x480, 30fps, 3Mbps).
Both the software and hardware decoders can decode at
the configured frame rate. The measurements also show that the 3G
module consumes additional 30%-45% of power.
To measure the impact of touch screens, we develop an
Android application that does nothing but accepting gestures from a user. The
accepted events are dropped immediately. We run this gesture application
on both mobile devices for 3 minutes and collect the CPU utilization and
power consumption. We continuously slide the touch screens during the second minute, and
leave the mobile devices idle in the first and last minutes. To isolate the
resource consumption due to touch screens and event processing, we also write
replayer application that injects the touch screen events to the gesture
application. We run the gesture application and log all the timestamped
events. We then inject the events to the gesture application using the replayer
application, and we compare the two measurement results.
Figure 8 gives the CPU utilization and power consumption over time. This
figure shows that the touch screen (including event processing) and event
processing (only) consume similar amount of resources. This indicates that the
event processing is energy-hungry, while the touch screen consumes negligible
amount of additional resources.
That is, touch screen and event processing is not free, although it may be
overlooked in the past.
Our measurements show that event processing consumes additional
6%-10% power.
More in-depth analysis along this line is among our future tasks.
Figure 7: CPU utilization and power consumption for different
wireless access links.
Figure 8: CPU utilization and power consumption for
handling control events.
5 Conclusion
In this paper, we implement a testbed using a real mobile cloud gaming
platform [10,11] developed by us. We conduct extensive experiments
to measure the client performance and energy consumption. Our measurement
results lead to the following main findings.
Running mobile cloud games is more energy efficient than native mobile games. Our experiments indicate that mobile cloud games reduce the CPU utilization by half, and save energy by 30%.
The hardware video decoders are not sensitive to higher video codec parameters, such as resolutions and bitrates. However, the hardware video decoders always buffer 1-2 frames, which are less ideal to fast-pace games.
The video codec parameters (bitrate, frame rate, and resolution) impose different degrees of impacts on CPU utilization and energy consumption. The frame rate affects the most, while the resolution affects the least.
Two other energy-hungry components are identified: (i) the 3G cellular module consumes 30%-45% more energy and (ii) the event processing of touch screens consumes nontrivial resources as well.
These insights lead to design recommendations for future researchers and
developers of the emerging mobile cloud gaming platforms.
References
[1]
W. Cai, C. Zhou, V. Leung, and M. Chen.
A cognitive platform for mobile cloud gaming.
In Proc. of the IEEE International Conference on Cloud Computing
Technology and Science (CloudCom'13), pages 72-79, Bristol, UK, December
2013.
[2]
K. Chen, Y. Chang, H. Hsu, D. Chen, C. Huang, and C. Hsu.
On The Quality of Service of Cloud Gaming Systems.
IEEE Transactions on Multimedia, 16(2), Feb 2014.
[3]
S. Chuah and N. Cheung.
Layered coding for mobile cloud gaming.
In Proc. of ACM International Workshop on Massively Multiuser
Virtual Environments (MMVE'14), pages 1-6, Singapore, March 2014.
[4]
Distribution and monetization strategies to increase revenues from cloud
gaming.
http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf">http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf.
[5]
Dragon Quest 10 going mobile in Japan with cloud-streaming service.
http://www.shacknews.com/article/81351/dragon-quest-10-going-mobile-in-japan-with-cloud-streaming">http://www.shacknews.com/article/81351/dragon-quest-10-going-mobile-in-japan-with-cloud-streaming.
[6]
Enhance your mobile gaming experience with the cloud.
http://www.verizonwireless.com/insiders-guide/entertainment/mobile-cloud-gaming/">http://www.verizonwireless.com/insiders-guide/entertainment/mobile-cloud-gaming/.
[7]
G.Perrucci, F. Fitzek, and J. Widmer.
Survey on energy consumption entities on the smartphone platform.
In Proc. of IEEE Vehicular Technology Conference (VTC
Spring'11), pages 1-6, Budapest, Hungary, May 2011.
[8]
R. Hans, U. Lampe, D. Burgstahler, M. Hellwig, and R. Steinmetz.
Where did my battery go? quantifying the energy consumption of cloud
gaming.
In Proc. of IEEE International Conference on Mobile Services
(MS'14), pages 63-67, Anchorage, AK, June 2014.
[9]
M. Hemmati, A. Javadtalab, A. Shirehjini, S. Shirmohammadi, and T. Arici.
Game as video: Bit rate reduction through adaptive object encoding.
In Proc. of ACM International Workshop on Network and Operating
Systems Support for Digital Audio and Video (NOSSDAV'13), pages 7-12, Oslo,
Norway, February 2013.
[10]
C.-Y. Huang, C.-H. Hsu, Y.-C. Chang, and K.-T. Chen.
GamingAnywhere: An Open Cloud Gaming System.
In Proc. of the ACM Multimedia Systems Conference (MMSys'13),
pages 36-47, Oslo, Norway, February 2013.
[11]
C.-Y. Huang, C.-H. Hsu, D.-Y. Chen, and K.-T. Chen.
Quantifying User Satisfaction in Mobile Cloud Games.
In Proceedings of ACM Workshop on Mobile Video Delivery
(MoVid'13), pages 4:1-4:6, 2014.
[12]
M. Jarschel, D. Schlosser, S. Scheuring, and T. Hobfeld.
An evaluation of QoE in cloud gaming based on subjective tests.
In Proc. of International Conference on Innovative Mobile and
Internet Services in Ubiquitous Computing (IMIS'11), pages 330-335, Seoul,
Korea, June 2011.
[13]
M. Jarschel, D. Schlosser, S. Scheuring, and T. Hobfeld.
Gaming in the clouds: QoE and the users' perspective.
Mathematical and Computer Modelling, 11-12(57):2883-2894, June
2013.
[14]
U. Lampe, R. Hans, and R. Steinmetz.
Will mobile cloud gaming work? Findings on latency, energy, and
cost.
In Proc. of IEEE International Conference on Mobile Services
(MS'13), pages 960-961, Santa Clara, CA, June 2013.
[15]
Y. Liu, S. Wang, and S. Dey.
Content-aware modeling and enhancing user experience in cloud mobile
rendering and streaming.
IEEE Journal on Emerging and Selected Topics in Circuits and
Systems, 4(1):43-56, March 2014.
[16]
P. Ross.
Cloud computing's killer app: Gaming.
IEEE Spectrum, 46(3):14, March 2009.
[17]
R. Shea, J. Liu, E. Ngai, and Y. Cui.
Cloud gaming: Architecture and performance.
IEEE Network Magazine, 27(4):16-21, July/August 2013.
[18]
S. Shi, C. Hsu, K. Nahrstedt, and R. Campbell.
Using graphics rendering contexts to enhance the real-time video
coding for mobile cloud gaming.
In Proc. of the ACM international conference on Multimedia
(MM'11), pages 103-112, Scottsdale, AZ, November 2011.
[19]
S. Wang and S. Dey.
Adaptive mobile cloud computing to enable rich mobile multimedia
applications.
IEEE Transactions on Multimedia, 15(4):870-883, June 2013.
Sheng-Wei Chen (also known as Kuan-Ta Chen) http://www.iis.sinica.edu.tw/~swc
Last Update September 28, 2019