Language Identification via Compositional Data Analysis: A Linear-Time Classifier Based on Log-Ratio Geometry
This paper proposes a method for language identification using compositional vectors and the centered log-ratio transformation, achieving robust accuracy and strong performance for longer sequences.
Proposes a new method for language identification using compositional vectors and the CLR transformation, offering a deterministic and computationally efficient alternative to neural approaches and statistical n-gram models.
Keywords
Before reading this…
Applications
- →Natural language processing
- →Speech recognition
- →Machine translation
To understand this paper, make sure you know these concepts first:
- Frequency distributionsfind papers →
- Centered log-ratio transformationfind papers →
- Dimensionality reductionfind papers →
Abstract
More Like ThisLanguage identification is commonly addressed using either neural architectures or statistical n-gram models. Neural approaches typically require substantial computational resources, whereas classical frequency-based methods offer efficient linear-time performance, but rely on distance metrics that are not always appropriate for compositional data. This work models character and bigram frequency distributions as compositional vectors constrained to the simplex and mapped via the centered log-ratio (CLR) transformation bijectively onto the $(D-1)$-dimensional zero-sum subspace of $\mathbb{R}^D$, where Euclidean distances correspond to Aitchison distances. A pipeline is proposed, combining CLR-transformed unigram and bigram features with Laplace smoothing to address sparsity. The method is evaluated on six languages. Experimental results show that the proposed approach achieves robust accuracy across different text lengths, with strong performance for longer sequences. These findings indicate that compositional representations provide a deterministic and computationally efficient alternative for language identification, particularly in settings where interpretability and low resource consumption are essential.