Looking for the Essence of Lexical Diversity
Lexical diversity (LD) is the variety of words in a text or a segment of speech. Scott Jarvis has provided evidence for needing an emic approach to provide the basis for understanding LD in several papers. As part of his work, he has got LD ratings from untrained human raters and showed that human raters have a common standard for LD. This thesis adds to his work in seeing if similar evidence can still be obtained for a common standard, while attempting to eliminate the possibility that Jarvis' findings could be an artifact of the way the data were collected. It also looks at whether human ratings of the individual properties of LD correlate significantly with LD itself.
This thesis utilizes twenty-one texts selected from a corpus. These texts provide the substance for raters to rate LD. The study has two groups of raters. One group rates LD and then all of the properties of LD. The other group rates the writing quality of the texts and then rates how much the author of the text they read likes seeing mountains, which is irrelevant to anything they have been given. Analyses are then performed to see if there are any general trends in the responses that could show LD or writing quality are treated in an equivalent manner to an irrelevant question. Furthermore, the standard deviations of LD, writing quality, and the irrelevant question are compared through the Friedman Rank Sum Test and post hoc Wilcoxon Signed-Rank Tests. The results suggest that LD and writing quality do not exhibit the same trends as an irrelevant question, suggesting that the ratings of LD and writing quality are meaningful. Additionally, they are more consistent than the irrelevant question, supporting that LD has a common standard and Jarvis’s results were not just an artifact of his study.
Further analyses include running various correlations between LD and each aspect of LD. These correlations were all found to be significant when the aspects of LD were correlated with their respective individual ratings of LD. However, volume, abundance, and rarity were no longer significant when the individual responses were condensed into a single rating for each text, which was done by averaging all of the ratings for each aspect and LD for each text. It is noted that the way volume and abundance were operationalized for human ratings in this study is likely the reason why they do not correlate significantly with LD in the averages. However, it is suggested that rarity may be related but it is not integral to LD.