LINGVISTIK VA MODAL AXBOROT OQIMINI STRUKTURALASH VA SEMANTIK MOSLIK ALGORITMLARI
https://doi.org/10.5281/zenodo.17241118
Keywords:
imo-ishora tili (IMO), multimodal axborot oqimi, gloss, semantik moslik, verifikatsiya algoritmi, UzSL, SLRAbstract
Ushbu maqolada multimodal imo-ishora oqimini strukturalash va semantik moslik algoritmlarini ishlab chiqish masalasi yoritiladi. IMO axborot oqimi gesture, pose va facial expression kabi modal elementlarning bir vaqtda qo‘llanishi tufayli murakkab xususiyatga ega. Shuning uchun uni lingvistik glosslar bilan uyg‘unlashtirish, vaqt bo‘yicha segmentlash va fazoviy-mazmuniy bog‘lanishlarni aniqlash dolzarb hisoblanadi.
Tadqiqotda semantik moslikni baholash mezonlari (xronologik muvofiqlik, modalitetlararo sinxronlik va glosslararo semantik constraint’lar) asosida verifikatsiya algoritmlari taklif etiladi. Xususan, glosslar orasidagi semantik yaqinlikni embedding-based scoring (SignBERT+), kontekstga asoslangan gloss tanlash (beam search, attention-filtered decoding) va ontologik tahlil (WordNet, ConceptNet, SignNet) usullari orqali baholash mexanizmlari ishlab chiqilgan.
Shuningdek, gloss-signal mapping algoritmlari tahlil qilinib, multimodal signallarni uzluksiz semantik oqimga aylantirish imkoniyatlari ko‘rib chiqiladi. Mahalliy imo-ishora tili korpuslariga tayangan holda UzSL uchun integratsiyalashgan strukturaviy-semantik model konsepsiyasi (UzSLNet) ishlab chiqish istiqbollari ham yoritilgan.
Olingan natijalar UzSL glosslarining tahlilini yanada samarali qilish, imo-ishora tilidagi signallarni kontekstdan kelib chiqib to‘g‘ri talqin qilish va xatoliklarni kamaytirishga xizmat qiladi. Bu esa sun’iy intellekt asosida ishonchli sign language recognition (SLR) platformalarini yaratish uchun nazariy va amaliy asos bo‘lib xizmat qiladi.
Downloads
References
Camgoz, N. C., Hadfield, S., Koller, O., & Bowden, R. (2018). Neural Sign Language Translation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 778–788. https://doi.org/10.1109/CVPR.2018.00086
Koller, O., Camgoz, N. C., Bowden, R., & Ney, H. (2020). Quantitative Survey of Sign Language Recognition. International Journal of Computer Vision, 128(2), 476–490. https://doi.org/10.1007/s11263-019-01168-9
Zhou, H., Zhou, W., Zhou, Y., & Li, H. (2021). Dynamic Graph Modeling for Sign Language Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10), 412–426. https://doi.org/10.1109/TPAMI.2020.2978384
Yin, K., Xia, H., Huenerfauth, M., & Liu, Y. (2021). SignBERT+: Handshape-Aware Sign Language Representation Learning. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), 541–550. https://doi.org/10.18653/v1/2021.acl-long.45
Chen, H., Liu, Y., Xu, R., & Fang, R. (2020). Sign Language Ontology for Semantic Mapping. Proceedings of the 12th International Conference on Language Resources and Evaluation (LREC), 2893–2901.
Li, X., & Zhao, Y. (2021). Multimodal Sign Language Processing: Foundations and Advances. Springer. https://doi.org/10.1007/978-3-030-78989-5
Hu, X., Zhao, R., & Li, H. (2023). Multimodal Aligner for Sign Language Recognition. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 1182–1194. https://doi.org/10.18653/v1/2023.acl-long.85
Zuo, Y., Huang, J., & Zhang, L. (2023). Graph-Temporal Alignment in Continuous Sign Language Recognition. IEEE Transactions on Multimedia, 25(6), 2510–2525. https://doi.org/10.1109/TMM.2023.3245671
Cihan Camgoz, N., Koller, O., Hadfield, S., & Bowden, R. (2020). Sign Language Transformers: Joint End-to-End Sign Language Recognition and Translation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10023–10033. https://doi.org/10.1109/CVPR42600.2020.01004
Xu, R., Wang, J., & Liu, Y. (2022). Multimodal Fusion for Sign Language Understanding. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 21(5), 1–18. https://doi.org/10.1145/3503044
Downloads
Published
Conference Proceedings Volume
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
