Statistical language models within the algebra of weighted rational languages
Statistical language models are an important tool in natural language processing. They represent prior knowledge about a certain language which is usually gained from a set of samples called a corpus. In this paper, we present a novel way of creating N-gram language models using weighted finite auto...
Elmentve itt :
| Szerzők: | |
|---|---|
| Testületi szerző: | |
| Dokumentumtípus: | Cikk |
| Megjelent: |
2009
|
| Sorozat: | Acta cybernetica
19 No. 2 |
| Kulcsszavak: | Számítástechnika, Kibernetika |
| Tárgyszavak: | |
| Online Access: | http://acta.bibl.u-szeged.hu/12868 |
| LEADER | 01618nab a2200241 i 4500 | ||
|---|---|---|---|
| 001 | acta12868 | ||
| 005 | 20220617090332.0 | ||
| 008 | 161015s2009 hu o 0|| eng d | ||
| 022 | |a 0324-721X | ||
| 040 | |a SZTE Egyetemi Kiadványok Repozitórium |b hun | ||
| 041 | |a eng | ||
| 100 | 1 | |a Hanneforth Thomas | |
| 245 | 1 | 0 | |a Statistical language models within the algebra of weighted rational languages |h [elektronikus dokumentum] / |c Hanneforth Thomas |
| 260 | |c 2009 | ||
| 300 | |a 313-356 | ||
| 490 | 0 | |a Acta cybernetica |v 19 No. 2 | |
| 520 | 3 | |a Statistical language models are an important tool in natural language processing. They represent prior knowledge about a certain language which is usually gained from a set of samples called a corpus. In this paper, we present a novel way of creating N-gram language models using weighted finite automata. The construction of these models is formalised within the algebra underlying weighted finite automata and expressed in terms of weighted rational languages and transductions. Besides the algebra we make use of five special constant weighted transductions which rely only on the alphabet and the model parameter N. In addition, we discuss efficient implementations of these transductions in terms of virtual constructions. | |
| 650 | 4 | |a Természettudományok | |
| 650 | 4 | |a Számítás- és információtudomány | |
| 695 | |a Számítástechnika, Kibernetika | ||
| 700 | 0 | 1 | |a Würzner Kay-Michael |e aut |
| 710 | |a Weighted Automata : Theory and Applications (2008) (Dresden) | ||
| 856 | 4 | 0 | |u http://acta.bibl.u-szeged.hu/12868/1/Hanneforth_2009_ActaCybernetica.pdf |z Dokumentum-elérés |