Multimodal Semantics Integration Using Ontologies Enhanced By Ontology Extraction And Cross Modality Disambiguation
The increasing amount of multimodal data such as text documents, annotated images and web pages have necessitated the development of effective techniques for their manipulation. The ineffectiveness of low-level image and textual features is one of the main issues as these features are commonly insuf...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2012
|
| Subjects: | |
| Online Access: | http://eprints.usm.my/41342/1/AHMAD_ADEL_AHMAD_ABU_SHAREHA.pdf http://eprints.usm.my/41342/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The increasing amount of multimodal data such as text documents, annotated images and web pages have necessitated the development of effective techniques for their manipulation. The ineffectiveness of low-level image and textual features is one of the main issues as these features are commonly insufficient for effective data manipulation. Therefore, obtaining sufficient and significant information from the multimodal data, and then further using this information in the proper manner is penultimate in data manipulation tasks. This thesis proposes a multimodal semantics integration (MSI) process to extract and integrate the semantics from the image and text modalities, and to use these semantics for manipulation tasks. The proposed process firstly extracts a textual representation from the textual and image modalities, followed by mapping the representation to concepts in a condensed knowledge source using a semantic-based alignment sub-process. |
|---|