Extracting Parallel Phrases from English-Punjabi Corpora

By: Lehal, Manpreet Singh

This study presents a novel approach to extract parallel data from a comparable English-Punjabi corpus, addressing the scarcity of parallel corpora for this language pair. Unlike previous research, this approach focuses on creating high-precision parallel data using minimal resources. The data is sourced from diverse domains, including Wikipedia articles, TDIL's noisy parallel sentences, and Gyan Nidhi reports. The methodology consists of three phases: extracting and aligning documents, translating Punjabi texts into English using OpenNMT-py, and calculating content similarity through three measures-Euclidean Distance, Cosine, and Jaccard. These algorithms are run individually, and then their results are integrated to improve accuracy. By combining the scores of all three measures, the system achieves a precision of 93% and an accuracy of 86%. This integrated approach significantly enhances parallel data extraction for English-Punjabi corpora and holds potential for improving Statistical Machine Translation (SMT) models.

• Format: Paperback
• Language: English
• ISBN: 6208225418
• ISBN13: 9786208225414
• Release Date: October 2024
• Publisher: LAP Lambert Academic Publishing
• Length: 204 Pages
• Weight: 0.67 lbs.
• Dimensions: 0.5" x 6.0" x 9.0"