A new deep CNN for 3D text localization in the wild through shadow removal
Text localization in the wild is challenging due to the presence of 2D and 3D texts, the presence of shadows, arbitrary orientated text with non-linear arrangements, varying lighting conditions as well as complex background. This paper proposes the first approach for 3D text localization in natural...
Saved in:
Main Authors: | , , , , , , |
---|---|
Format: | Article |
Published: |
Elsevier
2024
|
Subjects: | |
Online Access: | http://eprints.um.edu.my/44304/ https://doi.org/10.1016/j.cviu.2023.103863 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
id |
my.um.eprints.44304 |
---|---|
record_format |
eprints |
spelling |
my.um.eprints.443042024-07-05T02:57:05Z http://eprints.um.edu.my/44304/ A new deep CNN for 3D text localization in the wild through shadow removal Shivakumara, Palaiahnakote Banerjee, Ayan Nandanwar, Lokesh Pal, Umapada Antonacopoulos, Apostolos Lu, Tong Blumenstein, Michael QA75 Electronic computers. Computer science Text localization in the wild is challenging due to the presence of 2D and 3D texts, the presence of shadows, arbitrary orientated text with non-linear arrangements, varying lighting conditions as well as complex background. This paper proposes the first approach for 3D text localization in natural scene images through shadow removal and a new deep CNN model. In a first step, exploiting the observation that 3D text generates shadow information in natural scenes, the proposed model detects and removes the shadow pixels of 3D text based on the Generalized Gradient Vector Flow concept and a new clustering approach. The performance of the classification of 2D and 3D texts in the scene images is strengthened by using key features, including pixel strength, sharpness and edge potential, which are extracted to eliminate false text and shadow pixels. For text localization after removing shadow information, EfficientNet is used as an encoder (backbone) and UNet as a decoder in a novel way employing differential binarization. Experimental validation and comparative analysis with state-of-the-art approaches on both a new purpose-built dataset as well as on the benchmark datasets of ICDAR MLT 2019, ICDAR ArT 2019, CTW1500, DAST1500, Total-Text, and MSRATD500 for each of the different steps of the method, show that the proposed approach outperforms the existing methods. Elsevier 2024-01 Article PeerReviewed Shivakumara, Palaiahnakote and Banerjee, Ayan and Nandanwar, Lokesh and Pal, Umapada and Antonacopoulos, Apostolos and Lu, Tong and Blumenstein, Michael (2024) A new deep CNN for 3D text localization in the wild through shadow removal. Computer Vision and Image Understanding, 238. ISSN 1077-3142, DOI https://doi.org/10.1016/j.cviu.2023.103863 <https://doi.org/10.1016/j.cviu.2023.103863>. https://doi.org/10.1016/j.cviu.2023.103863 10.1016/j.cviu.2023.103863 |
institution |
Universiti Malaya |
building |
UM Library |
collection |
Institutional Repository |
continent |
Asia |
country |
Malaysia |
content_provider |
Universiti Malaya |
content_source |
UM Research Repository |
url_provider |
http://eprints.um.edu.my/ |
topic |
QA75 Electronic computers. Computer science |
spellingShingle |
QA75 Electronic computers. Computer science Shivakumara, Palaiahnakote Banerjee, Ayan Nandanwar, Lokesh Pal, Umapada Antonacopoulos, Apostolos Lu, Tong Blumenstein, Michael A new deep CNN for 3D text localization in the wild through shadow removal |
description |
Text localization in the wild is challenging due to the presence of 2D and 3D texts, the presence of shadows, arbitrary orientated text with non-linear arrangements, varying lighting conditions as well as complex background. This paper proposes the first approach for 3D text localization in natural scene images through shadow removal and a new deep CNN model. In a first step, exploiting the observation that 3D text generates shadow information in natural scenes, the proposed model detects and removes the shadow pixels of 3D text based on the Generalized Gradient Vector Flow concept and a new clustering approach. The performance of the classification of 2D and 3D texts in the scene images is strengthened by using key features, including pixel strength, sharpness and edge potential, which are extracted to eliminate false text and shadow pixels. For text localization after removing shadow information, EfficientNet is used as an encoder (backbone) and UNet as a decoder in a novel way employing differential binarization. Experimental validation and comparative analysis with state-of-the-art approaches on both a new purpose-built dataset as well as on the benchmark datasets of ICDAR MLT 2019, ICDAR ArT 2019, CTW1500, DAST1500, Total-Text, and MSRATD500 for each of the different steps of the method, show that the proposed approach outperforms the existing methods. |
format |
Article |
author |
Shivakumara, Palaiahnakote Banerjee, Ayan Nandanwar, Lokesh Pal, Umapada Antonacopoulos, Apostolos Lu, Tong Blumenstein, Michael |
author_facet |
Shivakumara, Palaiahnakote Banerjee, Ayan Nandanwar, Lokesh Pal, Umapada Antonacopoulos, Apostolos Lu, Tong Blumenstein, Michael |
author_sort |
Shivakumara, Palaiahnakote |
title |
A new deep CNN for 3D text localization in the wild through shadow removal |
title_short |
A new deep CNN for 3D text localization in the wild through shadow removal |
title_full |
A new deep CNN for 3D text localization in the wild through shadow removal |
title_fullStr |
A new deep CNN for 3D text localization in the wild through shadow removal |
title_full_unstemmed |
A new deep CNN for 3D text localization in the wild through shadow removal |
title_sort |
new deep cnn for 3d text localization in the wild through shadow removal |
publisher |
Elsevier |
publishDate |
2024 |
url |
http://eprints.um.edu.my/44304/ https://doi.org/10.1016/j.cviu.2023.103863 |
_version_ |
1805881154750906368 |
score |
13.19449 |