Skip Nav Destination
Close Modal
Update search
NARROW
Format
Journal
TocHeadingTitle
Date
Availability
1-8 of 8
Richard Sproat
Close
Follow your search
Access your saved searches in your account
Would you like to receive an alert when new items match your search?
Sort by
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2022) 48 (2): 483–490.
Published: 09 June 2022
Abstract
View articletitled, Boring Problems Are Sometimes the Most Interesting
View
PDF
for article titled, Boring Problems Are Sometimes the Most Interesting
In a recent position paper, Turing Award Winners Yoshua Bengio, Geoffrey Hinton, and Yann LeCun make the case that symbolic methods are not needed in AI and that, while there are still many issues to be resolved, AI will be solved using purely neural methods. In this piece I issue a challenge: Demonstrate that a purely neural approach to the problem of text normalization is possible. Various groups have tried, but so far nobody has eliminated the problem of unrecoverable errors , errors where, due to insufficient training data or faulty generalization, the system substitutes some other reading for the correct one. Solutions have been proposed that involve a marriage of traditional finite-state methods with neural models, but thus far nobody has shown that the problem can be solved using neural methods alone. Though text normalization is hardly an “exciting” problem, I argue that until one can solve “boring” problems like that using purely AI methods, one cannot claim that AI is a success.
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2021) 47 (3): 477–528.
Published: 03 November 2021
FIGURES
| View All (14)
Abstract
View articletitled, The Taxonomy of Writing Systems: How to Measure How Logographic a System Is
View
PDF
for article titled, The Taxonomy of Writing Systems: How to Measure How Logographic a System Is
Taxonomies of writing systems since Gelb ( 1952 ) have classified systems based on what the written symbols represent: if they represent words or morphemes, they are logographic ; if syllables, syllabic ; if segments, alphabetic ; and so forth. Sproat ( 2000 ) and Rogers ( 2005 ) broke with tradition by splitting the logographic and phonographic aspects into two dimensions, with logography being graded rather than a categorical distinction. A system could be syllabic, and highly logographic; or alphabetic, and mostly non-logographic. This accords better with how writing systems actually work, but neither author proposed a method for measuring logography. In this article we propose a novel measure of the degree of logography that uses an attention-based sequence-to-sequence model trained to predict the spelling of a token from its pronunciation in context. In an ideal phonographic system, the model should need to attend to only the current token in order to compute how to spell it, and this would show in the attention matrix activations. In contrast, with a logographic system, where a given pronunciation might correspond to several different spellings, the model would need to attend to a broader context. The ratio of the activation outside the token and the total activation forms the basis of our measure. We compare this with a simple lexical measure, and an entropic measure, as well as several other neural models, and argue that on balance our attention-based measure accords best with intuition about how logographic various systems are. Our work provides the first quantifiable measure of the notion of logography that accords with linguistic intuition and, we argue, provides better insight into what this notion means.
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2019) 45 (2): 293–337.
Published: 01 June 2019
FIGURES
| View All (10)
Abstract
View articletitled, Neural Models of Text Normalization for Speech Applications
View
PDF
for article titled, Neural Models of Text Normalization for Speech Applications
Machine learning, including neural network techniques, have been applied to virtually every domain in natural language processing. One problem that has been somewhat resistant to effective machine learning solutions is text normalization for speech applications such as text-to-speech synthesis (TTS). In this application, one must decide, for example, that 123 is verbalized as one hundred twenty three in 123 pages but as one twenty three in 123 King Ave. For this task, state-of-the-art industrial systems depend heavily on hand-written language-specific grammars. We propose neural network models that treat text normalization for TTS as a sequence-to-sequence problem, in which the input is a text token in context, and the output is the verbalization of that token. We find that the most effective model, in accuracy and efficiency, is one where the sentential context is computed once and the results of that computation are combined with the computation of each token in sequence to compute the verbalization. This model allows for a great deal of flexibility in terms of representing the context, and also allows us to integrate tagging and segmentation into the process. These models perform very well overall, but occasionally they will predict wildly inappropriate verbalizations, such as reading 3 cm as three kilometers . Although rare, such verbalizations are a major issue for TTS applications. We thus use finite-state covering grammars to guide the neural models, either during training and decoding, or just during decoding, away from such “unrecoverable” errors. Such grammars can largely be learned from data.
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2014) 40 (4): 733–761.
Published: 01 December 2014
FIGURES
| View All (14)
Abstract
View articletitled, Applications of Lexicographic Semirings to Problems in Speech and Language Processing
View
PDF
for article titled, Applications of Lexicographic Semirings to Problems in Speech and Language Processing
This paper explores lexicographic semirings and their application to problems in speech and language processing. Specifically, we present two instantiations of binary lexicographic semirings, one involving a pair of tropical weights, and the other a tropical weight paired with a novel string semiring we term the categorial semiring . The first of these is used to yield an exact encoding of backoff models with epsilon transitions. This lexicographic language model semiring allows for off-line optimization of exact models represented as large weighted finite-state transducers in contrast to implicit (on-line) failure transition representations. We present empirical results demonstrating that, even in simple intersection scenarios amenable to the use of failure transitions, the use of the more powerful lexicographic semiring is competitive in terms of time of intersection. The second of these lexicographic semirings is applied to the problem of extracting, from a lattice of word sequences tagged for part of speech, only the single best-scoring part of speech tagging for each word sequence. We do this by incorporating the tags as a categorial weight in the second component of a 〈Tropical, Categorial〉 lexicographic semiring, determinizing the resulting word lattice acceptor in that semiring, and then mapping the tags back as output labels of the word lattice transducer. We compare our approach to a competing method due to Povey et al. (2012).
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2010) 36 (4): 807–816.
Published: 01 December 2010
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2010) 36 (3): 585–594.
Published: 01 September 2010
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2008) 34 (4): 615–617.
Published: 01 December 2008
View articletitled, Mathematical Linguistics András Kornai (MetaCarta Inc.) Springer (Advanced information and knowledge processing series, edited by Lakhmi Jain), 2008, xiii+289 pp; ISBN 978-1-84628-985-9, $99.00
View
PDF
for article titled, Mathematical Linguistics András Kornai (MetaCarta Inc.) Springer (Advanced information and knowledge processing series, edited by Lakhmi Jain), 2008, xiii+289 pp; ISBN 978-1-84628-985-9, $99.00
Journal Articles
Publisher: Journals Gateway
Computational Linguistics (2001) 27 (3): 450–456.
Published: 01 September 2001