Tesis

Fine-tuning multilingual transformers for automatic assessment of cross-language concept map semantic similarity / Nadindra Dwi Ariyanta

Abstrak
This study evaluates the performance of multilingual transformer models in automated cross-lingual concept map assessment addressing inherent challenges in increasingly diverse global educational environments. While concept maps are crucial tools for student comprehension their automated assessment is complex particularly as pre-trained models like Multilingual Bidirectional Encoder Representations from Transformers (mBERT) Cross-Lingual Language Model - RoBERTa (XLM-R) Multilingual Bidirectional and Auto-Regressive Transformers (mBART) and Multilingual Text-to-Text Transfer Transformer (MT5) are often not optimally suited for this specific semantic similarity task especially in low-resource languages. Therefore this research aims to evaluate the performance of these models in their pre-trained state and analyze the impact of fine-tuning on their ability to automatically assess concept map quality. The methodology involved collecting Indonesian and English concept map data followed by data preprocessing (cleaning case folding tokenization) and fine-tuning the models on STS-B (Semantic Textual Similarity Benchmark) and GLUE (General Language Understanding Evaluation) datasets. Semantic similarity was measured using Cosine Similarity and model performance was evaluated with Accuracy Precision F1-Score RMSE and MAE. Results indicate that fine-tuning significantly improved performance for Indonesian (e.g. fine-tuned XLM-R and mBART achieved 89% accuracy and 94% F1-score) though overfitting was identified. Performance improvement in English was less significant due to its inherent linguistic complexity. The study s implications underscore the importance of targeted fine-tuning to maximize model effectiveness while also highlighting the need for more efficient fine-tuning strategies to mitigate overfitting and enhance generalization on complex datasets.

Informasi Detail

DDC

Rt 621.3076 ARI f

Prodi

Universitas Negeri Malang. Program Studi Teknik Elektro, 2025.

Deskripsi Fisik

x, 41 lembar. : ilus. ; 30 cm.

Bahasa

No Reg

00638/RT/25

Edisi

Tesis (Pascasarjana)--Universitas Negeri Malang. 2025

Subjek

1. TEKNIK ELEKTRO - TRANSFORMASI MULTIBAHASA
2. PETA KONSEP LINTAS BAHASA
3. ELECTRICAL ENGINEERING - MULTILINGUAL TRANSFORMATION

Pembimbing

1. Dr.eng Didik Dwi Prasetya, S.t., M.t.; 2. Ilham Ari Elbaith Zaeni, S.t., M.t., Ph.d

Lampiran Berkas