PIs: Mi-Yen Yeh, De-Nian Yang, Shou-De Lin, Chih-Ya Shen, and Hong-Han Shuai
Our project aims to investigate smart digital marketing and social metaverse, including MVD-enabled XR shopping system, coupled user and item recommendations for avatars with conversation generation, dynamic influence model for correlated items and user perception, royalty and resale profit maximization on NFTs, generalized winning price model for display slots bidding, adversarial attacks for metaverse with heterogeneous GNN, SIoT deployment for real-time detection and computation, network optimization for dual entangled worlds, and unbiased learning. The project involves industrial collaboration with Innolux, Chunghwa Telecom, Appier, China Engineering Consultants (CECI), Institute for Information Industry (III), ETtoday, Kbro Broadband, MyVideo, and E.SUN Bank. For instance, with Innolux, we leverage knowledge graphs (KGs) to explore the dependence of defects-related factors and identify a group of critical factors to improve the defect rate; with CECI, we recommend companions and POIs to satisfy user preferences and social needs, while ensuring a low infection risk and avoiding traffic congestion; with III, we investigate sequential predictions on evolving KGs by modeling the local relations and global trends to generate a sequence of future events; with E.SUN Bank, we analyze customers’ interactions with chatbots and other customers to build social networks and predict customer intent to provide timely assistance. The project organized two special issues in IEEE Communications Magazine and IEEE Transactions on Computational Social Systems, coordinated DASFAA 2021, and will host PAKDD 2024. The research results have been accepted by top conferences and international journals, including ACM SIGKDD, IEEE ICDE, VLDB, WWW, ACM CIKM, IEEE BigData, NeurIPS, AAAI, CVPR, ICCV, IEEE INFOCOM, IEEE TKDE, ACM TKDD, IEEE TBD, IEEE TSC, IEEE TMC, IEEE IoTJ, and IEEE TPDS.
Social Group Queries and Influence Maximization for Digital Marketing.
Digital marketing campaigns can be facilitated by social group queries (SGQs) and influence maximization (IM). Nevertheless, conventional group queries only extract dense subgraphs and fail to specify parameters for diverse needs; traditional IM does not distribute social coupons (SCs) and frequently results in disparity between under-represented groups. To address these issues, we develop the world’s first parameter-free SGQs to select users and find items, locations, and available time to maximize the opportunity of building potential links (e.g., co-purchase), and then propose the premier campaign for multiple items across multiple promotions by seeding users and distributing SCs to maximize SC redemption and influence while avoiding discrimination. For the hardness analysis, we prove that the problems are NP-Hard since they consider changes in item relationships, user preferences, and social influence by leveraging KGs, while prior works only promote a single item in a single promotion.
To tackle the problems, we first utilize graph neural networks to extract node features from the heterogeneous activity information network and design a heterogeneous information network transformer for learning embeddings to infer diverse user preferences based on node features and potential links, whereas previous approaches only account for the current topology. We also develop graph memory refreshing, the first theoretical guarantee of embedding quality, to efficiently update embeddings under multi-type network changes. To find social groups and items, we propose efficient algorithms using inferred user preferences for the optimal solution to a sequence of activities subject to various spatiotemporal constraints and also design an error-bounded 3-approximation algorithm for sufficiently large and tight groups. Based on the social groups and items, we are the pioneer in leveraging KGs to develop approximation IM algorithms for selecting promoting items and seeds in collaboration with SCs. Our novel community-gender-aware seeding framework and two gender-aware metrics prioritize influential users without discrimination. Compared to Airbnb and Booking.com policies, our approaches increase the redemption rate by up to 30 times. We have collaborated with CECI (Taiwan’s leading transportation engineering consulting firm) for social-aware advertising, which dynamically customizes ads and coupons for different users based on their locations, preferences, social influence, and real-time feedback, in contrast to out-of-home advertising (e.g., More Media), which displays the same content for everyone. The results have been published in ACM KDD, WWW, VLDB, IEEE ICDE, AAAI, ACM CIKM, IEEE TKDE, and IEEE TBD.
Data Mining and Database Query in Social Metaverse.
The social metaverse is a new trend in social media, as evidenced by Meta Horizon, Microsoft AltSpaceVR, and Mozilla Hubs. Existing VR shopping in the metaverse, such as IKEA VR Store and eBay and Myer’s VR department store, is designed for single users; traditional spatial queries are inadequate for the immersive experience of the metaverse. With the support of multi-view display (MVD), digital twin, and live multi-streaming, we envision VR-based group shopping in the social metaverse, namely MVD-enabled VR Group-Item and User Configuration, Donation Recommendation, and Dual-World Obstacle-free Path Query in Social Metaverse. VR-based group shopping in the social metaverse addresses individual preferences more effectively due to display flexibility; it also increases sales and customer satisfaction via co-presence and immersion, in contrast to e-commerce that emphasizes personalized recommendations. We are the first metaverse recommendation system to break the boundary between personalized and group recommendations. We prove that it is APX-hard and NP-hard to approximate, since it involves real-time social interactions, the socio-temporal ripple effect of donations, and content diversity, while prior recommendations only focus on preferences, and that personalized and group recommendations are our special cases.
To deal with the problem, we first extract features and learn MVD configurations from multiple feedback by configuration feature-coupled tensor and socio-temporal donation-response tensor, and infer personal preferences using a digital twin attractiveness autoencoder based on extracted features. Then, we design a 4-approximation algorithm to configure displayed items and recommend suitable users for different subgroups according to extracted features and user preferences to maximize item diversity and feedback quality while taking co-display and occlusion-aware attractiveness into account. To reach displayed items and users, we devise a fully polynomial-time approximation scheme for determining the minimum immersion loss range for obstacle-free paths. This is the first immersive spatial query in the metaverse and a cornerstone for supporting kNN and range queries in the dual worlds. We have implemented recommendation tools on Twitch (the largest live-streaming platform worldwide) and with Unity, Photon Unity Network, Steam VR Plugin, VRTK, and 3ds Max for users wearing hTC VIVE and Meta Oculus Quest 2 HMD. The results have been published in VLDB, AAAI, ACM CIKM, NeurIPS, CVPR, ICCV, and IEEE TMC.
Epidemic-aware Digital Marketing.
During times of crisis, such as the COVID-19 pandemic, businesses and marketers must adapt their digital marketing strategies to account for changes in consumer preferences, behavior, and priorities. However, current public health studies aim to capture epidemic dynamics and assess policy effectiveness without planning containment measures or recommending specific points of interest (POIs). To address this gap, we present the first work on epidemic-aware POI recommendations and precise containment with Non-Pharmaceutical Intervention (NPI) and vaccination by analyzing the contact social network. Due to entangled objectives, including epidemic containment, user satisfaction maximization, and adverse effect minimization, we prove that it is NP-hard and inapproximable within any ratio. To solve the problem, we are the trailblazer in proposing an approximation algorithm to schedule NPIs and vaccines for different groups at different times for epidemic containment while reducing negative impacts. Based on the containment operation, we then develop an epidemic-aware POI recommendation algorithm to suggest POIs and companions that not only comply with the containment actions but also protect vulnerable users as well as prevent superspreaders, whereas prior epidemic containment algorithms do not recommend POIs while controlling the epidemic and previous POI recommendations and social-spatial-temporal queries cannot guarantee epidemic control. We have assisted the local government in coordinating the distribution of medical supplies to diverse populations and regions. The results have been published in IEEE ICDE and IEEE MDM.