Paragraph specific n-gram approaches to automatically assessing essay quality

Scott Crossley, Caleb DeFore, Kris Kyle, Jianmin Dai, Danielle S. McNamara

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

In this paper, we describe an n-gram approach to automatically assess essay quality in student writing. Underlying this approach is the development of n-gram indices that examine rhetorical, syntactic, grammatical, and cohesion features of paragraph types (introduction, body, and conclusion paragraphs) and entire essays. For this study, we developed over 300 n-gram indices and assessed their potential to predict human ratings of essay quality. A combination of these n-gram indices explained over 30% of the variance in human ratings for essays in a training and testing corpus. The findings from this study indicate the strength of using n-gram indices to automatically assess writing quality. Such indices not only explain text-based factors that influence human judgments of essay quality, but also provide new methods for automatically assessing writing quality.

Original languageEnglish
Title of host publicationProceedings of the 6th International Conference on Educational Data Mining, EDM 2013
EditorsSidney K. D'Mello, Rafael A. Calvo, Andrew Olney
PublisherInternational Educational Data Mining Society
ISBN (Electronic)9780983952527
Publication statusPublished - 2013 Jan 1
Event6th International Conference on Educational Data Mining, EDM 2013 - Memphis, United States
Duration: 2013 Jul 62013 Jul 9

Publication series

NameProceedings of the 6th International Conference on Educational Data Mining, EDM 2013

Conference

Conference6th International Conference on Educational Data Mining, EDM 2013
CountryUnited States
CityMemphis
Period13/7/613/7/9

Fingerprint

Syntactics
Students
Testing

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Information Systems

Cite this

Crossley, S., DeFore, C., Kyle, K., Dai, J., & McNamara, D. S. (2013). Paragraph specific n-gram approaches to automatically assessing essay quality. In S. K. D'Mello, R. A. Calvo, & A. Olney (Eds.), Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013 (Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013). International Educational Data Mining Society.
Crossley, Scott ; DeFore, Caleb ; Kyle, Kris ; Dai, Jianmin ; McNamara, Danielle S. / Paragraph specific n-gram approaches to automatically assessing essay quality. Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013. editor / Sidney K. D'Mello ; Rafael A. Calvo ; Andrew Olney. International Educational Data Mining Society, 2013. (Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013).
@inproceedings{bc82f08af2384793bf1c8ffdb06a9576,
title = "Paragraph specific n-gram approaches to automatically assessing essay quality",
abstract = "In this paper, we describe an n-gram approach to automatically assess essay quality in student writing. Underlying this approach is the development of n-gram indices that examine rhetorical, syntactic, grammatical, and cohesion features of paragraph types (introduction, body, and conclusion paragraphs) and entire essays. For this study, we developed over 300 n-gram indices and assessed their potential to predict human ratings of essay quality. A combination of these n-gram indices explained over 30{\%} of the variance in human ratings for essays in a training and testing corpus. The findings from this study indicate the strength of using n-gram indices to automatically assess writing quality. Such indices not only explain text-based factors that influence human judgments of essay quality, but also provide new methods for automatically assessing writing quality.",
author = "Scott Crossley and Caleb DeFore and Kris Kyle and Jianmin Dai and McNamara, {Danielle S.}",
year = "2013",
month = "1",
day = "1",
language = "English",
series = "Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013",
publisher = "International Educational Data Mining Society",
editor = "D'Mello, {Sidney K.} and Calvo, {Rafael A.} and Andrew Olney",
booktitle = "Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013",

}

Crossley, S, DeFore, C, Kyle, K, Dai, J & McNamara, DS 2013, Paragraph specific n-gram approaches to automatically assessing essay quality. in SK D'Mello, RA Calvo & A Olney (eds), Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013. Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013, International Educational Data Mining Society, 6th International Conference on Educational Data Mining, EDM 2013, Memphis, United States, 13/7/6.

Paragraph specific n-gram approaches to automatically assessing essay quality. / Crossley, Scott; DeFore, Caleb; Kyle, Kris; Dai, Jianmin; McNamara, Danielle S.

Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013. ed. / Sidney K. D'Mello; Rafael A. Calvo; Andrew Olney. International Educational Data Mining Society, 2013. (Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Paragraph specific n-gram approaches to automatically assessing essay quality

AU - Crossley, Scott

AU - DeFore, Caleb

AU - Kyle, Kris

AU - Dai, Jianmin

AU - McNamara, Danielle S.

PY - 2013/1/1

Y1 - 2013/1/1

N2 - In this paper, we describe an n-gram approach to automatically assess essay quality in student writing. Underlying this approach is the development of n-gram indices that examine rhetorical, syntactic, grammatical, and cohesion features of paragraph types (introduction, body, and conclusion paragraphs) and entire essays. For this study, we developed over 300 n-gram indices and assessed their potential to predict human ratings of essay quality. A combination of these n-gram indices explained over 30% of the variance in human ratings for essays in a training and testing corpus. The findings from this study indicate the strength of using n-gram indices to automatically assess writing quality. Such indices not only explain text-based factors that influence human judgments of essay quality, but also provide new methods for automatically assessing writing quality.

AB - In this paper, we describe an n-gram approach to automatically assess essay quality in student writing. Underlying this approach is the development of n-gram indices that examine rhetorical, syntactic, grammatical, and cohesion features of paragraph types (introduction, body, and conclusion paragraphs) and entire essays. For this study, we developed over 300 n-gram indices and assessed their potential to predict human ratings of essay quality. A combination of these n-gram indices explained over 30% of the variance in human ratings for essays in a training and testing corpus. The findings from this study indicate the strength of using n-gram indices to automatically assess writing quality. Such indices not only explain text-based factors that influence human judgments of essay quality, but also provide new methods for automatically assessing writing quality.

UR - http://www.scopus.com/inward/record.url?scp=84911912617&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84911912617&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84911912617

T3 - Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013

BT - Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013

A2 - D'Mello, Sidney K.

A2 - Calvo, Rafael A.

A2 - Olney, Andrew

PB - International Educational Data Mining Society

ER -

Crossley S, DeFore C, Kyle K, Dai J, McNamara DS. Paragraph specific n-gram approaches to automatically assessing essay quality. In D'Mello SK, Calvo RA, Olney A, editors, Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013. International Educational Data Mining Society. 2013. (Proceedings of the 6th International Conference on Educational Data Mining, EDM 2013).