Researchers Develop High-Accuracy and Explainable Models for Vocabulary Difficulty Prediction

Researchers have developed two models for predicting vocabulary difficulty, achieving high accuracy and explainability in a shared task at the 2024 Workshop on Evaluating Economic and Societal Impact of Language Technologies (BEA). The team reported a black-box model with a correlation coefficient (r) exceeding 0.91 and an explainable model surpassing a fine-tuned encoder baseline with r > 0.77, according to a preprint published on arXiv.

The black-box model, based on a fine-tuned large language model (LLM) using a soft-target loss function, secured top results in the open track of the shared task. The explainable model analyzed factors such as spelling complexity and word structure from the British Council’s Knowledge of Vocabulary Lexicon (KVL) dataset. The study marks an important development in balancing predictive accuracy with interpretability in lexical difficulty assessment.

The BEA 2024 shared task focuses on advancing methods to quantify vocabulary difficulty, an important area for language education and computational linguistics. While the black-box model’s performance demonstrates the potential of LLMs in rating tasks, the explainable model provides insights into linguistic features affecting difficulty judgments.

Citations: arXiv:2605.14257v1 (accessed 2024-05-14)