This research explores improving the performance of Indonesian natural language processing (NLP) benchmarks, focusing on emotion classification, sentiment analysis, and textual entailment tasks. Utilizing the IndoBERT, a Transformer-based model tailored for the Indonesian language, the study employs a hybrid approach that combines the modified IndoBERT model — using the sum of its last four layers — with neural network architectures such as the bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent unit (BiGRU), and attention layers. The performance of these models is evaluated using the F1-score, offering insights into the effectiveness of layer modification and hybridization in addressing Indonesian NLP challenges.
The experimental results reveal that the hybrid IndoBERT-BiLSTM model consistently outperforms others, achieving an F1 Score of 0.93 for sentiment analysis and 0.78 for emotion classification. For textual entailment, the IndoBERT-BiGRU hybrid model leads with an F1 Score of 0.87, followed closely by the IndoBERT-BiLSTM combination at 0.84. These findings highlight the potential of Transformer-based hybrid models to enhance Indonesian NLP tasks, demonstrating significant improvements over existing benchmarks and paving the way for more sophisticated language understanding models.
For more details of our project contribution can be found at:
