Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework
Scalable link prediction in social networks allow dynamic social interaction gathering, potential friend suggestions, and community detection. Distributed open-source frameworks such as Hadoop and Spark facilitate efficient link prediction especially in large-scale social networks. The frameworks pr...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | Article |
Published: |
Elsevier
2022
|
Subjects: | |
Online Access: | http://eprints.um.edu.my/41131/ |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
id |
my.um.eprints.41131 |
---|---|
record_format |
eprints |
spelling |
my.um.eprints.411312023-09-07T06:33:23Z http://eprints.um.edu.my/41131/ Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework Daud, Nur Nasuha Ab Hamid, Siti Hafizah Saadoon, Muntadher Seri, Chempaka Hasan, Zati Hakim Azizul Anuar, Nor Badrul QA75 Electronic computers. Computer science Scalable link prediction in social networks allow dynamic social interaction gathering, potential friend suggestions, and community detection. Distributed open-source frameworks such as Hadoop and Spark facilitate efficient link prediction especially in large-scale social networks. The frameworks provide different kinds of tunable properties for users to manually configure the parameters for the applications. However, manual configurations are open to performance issues when the applications start scaling tremendously, which are hard to set up and are exposed to human errors. This paper proposes a novel Self-Configured Framework (SCF) to provide an autonomous feature in Spark that predicts and sets the best configuration instantly before the application execution using the XGBoost classifier. The framework with a self-configuration setting demonstrates a 40% reduction in prediction time as well as a balanced resource consumption that makes full use of resources, especially for limited number and size of clusters. The presented framework establishes its efficiency for link prediction in large-scale social networks by automatically configuring the best configuration suitable for a specific application given the varying dataset size of the Twitter social network, workload, and cluster specification. (C) 2022 Elsevier B.V. All rights reserved. Elsevier 2022-11 Article PeerReviewed Daud, Nur Nasuha and Ab Hamid, Siti Hafizah and Saadoon, Muntadher and Seri, Chempaka and Hasan, Zati Hakim Azizul and Anuar, Nor Badrul (2022) Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework. Knowledge-Based Systems, 255. ISSN 0950-7051, DOI https://doi.org/10.1016/j.knosys.2022.109713 <https://doi.org/10.1016/j.knosys.2022.109713>. 10.1016/j.knosys.2022.109713 |
institution |
Universiti Malaya |
building |
UM Library |
collection |
Institutional Repository |
continent |
Asia |
country |
Malaysia |
content_provider |
Universiti Malaya |
content_source |
UM Research Repository |
url_provider |
http://eprints.um.edu.my/ |
topic |
QA75 Electronic computers. Computer science |
spellingShingle |
QA75 Electronic computers. Computer science Daud, Nur Nasuha Ab Hamid, Siti Hafizah Saadoon, Muntadher Seri, Chempaka Hasan, Zati Hakim Azizul Anuar, Nor Badrul Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
description |
Scalable link prediction in social networks allow dynamic social interaction gathering, potential friend suggestions, and community detection. Distributed open-source frameworks such as Hadoop and Spark facilitate efficient link prediction especially in large-scale social networks. The frameworks provide different kinds of tunable properties for users to manually configure the parameters for the applications. However, manual configurations are open to performance issues when the applications start scaling tremendously, which are hard to set up and are exposed to human errors. This paper proposes a novel Self-Configured Framework (SCF) to provide an autonomous feature in Spark that predicts and sets the best configuration instantly before the application execution using the XGBoost classifier. The framework with a self-configuration setting demonstrates a 40% reduction in prediction time as well as a balanced resource consumption that makes full use of resources, especially for limited number and size of clusters. The presented framework establishes its efficiency for link prediction in large-scale social networks by automatically configuring the best configuration suitable for a specific application given the varying dataset size of the Twitter social network, workload, and cluster specification. (C) 2022 Elsevier B.V. All rights reserved. |
format |
Article |
author |
Daud, Nur Nasuha Ab Hamid, Siti Hafizah Saadoon, Muntadher Seri, Chempaka Hasan, Zati Hakim Azizul Anuar, Nor Badrul |
author_facet |
Daud, Nur Nasuha Ab Hamid, Siti Hafizah Saadoon, Muntadher Seri, Chempaka Hasan, Zati Hakim Azizul Anuar, Nor Badrul |
author_sort |
Daud, Nur Nasuha |
title |
Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
title_short |
Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
title_full |
Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
title_fullStr |
Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
title_full_unstemmed |
Self-configured framework for scalable link prediction in twitter: Towards autonomous spark framework |
title_sort |
self-configured framework for scalable link prediction in twitter: towards autonomous spark framework |
publisher |
Elsevier |
publishDate |
2022 |
url |
http://eprints.um.edu.my/41131/ |
_version_ |
1778161629852073984 |
score |
13.160551 |