Assessing syntactic sophistication in L2 writing: A usage-based approach

Kristopher Kyle, Scott Crossley

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Over the past 45 years, the construct of syntactic sophistication has been assessed in L2 writing using what Bulté and Housen (2012) refer to as absolute complexity (Lu, 2011; Ortega, 2003; Wolfe-Quintero, Inagaki, & Kim, 1998). However, it has been argued that making inferences about learners based on absolute complexity indices (e.g., mean length of t-unit and mean length of clause) may be difficult, both from practical and theoretical perspectives (Norris & Ortega, 2009). Furthermore, indices of absolute complexity may not align with some prominent theories of language learning such as usage-based theories (e.g., Ellis, 2002a,b). This study introduces a corpus-based approach for measuring syntactic sophistication in L2 writing using a usage-based, frequency-driven perspective. Specifically, novel computational indices related to the frequency of verb argument constructions (VACs) and the strength of association between VACs and the verbs that fill them (i.e., verb–VAC combinations) are developed. These indices are then compared against traditional indices of syntactic complexity (e.g., mean length of T-unit and mean length of clause) with regard to their ability to model one aspect of holistic scores of writing quality in Test of English as a Foreign Language (TOEFL) independent essays. Indices related to usage-based theories of syntactic development explained greater variance (R2 =.142) in holistic scores of writing quality than traditional methods of assessing syntactic complexity (R2 =.058). The results have important implications for modeling syntactic sophistication, L2 writing assessment, and AES systems.

Original languageEnglish
Pages (from-to)513-535
Number of pages23
JournalLanguage Testing
Volume34
Issue number4
DOIs
Publication statusPublished - 2017 Oct 1

Fingerprint

theory formation
foreign language
L2 Writing
Syntax
Sophistication
Usage-based
Length
ability
language
learning
Verbs
Syntactic Complexity
Clause
Computational
Language Acquisition
Syntactic Development
English as a Foreign Language
Corpus-based
Writing Assessment
Modeling

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language

Cite this

@article{3a273384bede4124b4e87467bfdaf61f,
title = "Assessing syntactic sophistication in L2 writing: A usage-based approach",
abstract = "Over the past 45 years, the construct of syntactic sophistication has been assessed in L2 writing using what Bult{\'e} and Housen (2012) refer to as absolute complexity (Lu, 2011; Ortega, 2003; Wolfe-Quintero, Inagaki, & Kim, 1998). However, it has been argued that making inferences about learners based on absolute complexity indices (e.g., mean length of t-unit and mean length of clause) may be difficult, both from practical and theoretical perspectives (Norris & Ortega, 2009). Furthermore, indices of absolute complexity may not align with some prominent theories of language learning such as usage-based theories (e.g., Ellis, 2002a,b). This study introduces a corpus-based approach for measuring syntactic sophistication in L2 writing using a usage-based, frequency-driven perspective. Specifically, novel computational indices related to the frequency of verb argument constructions (VACs) and the strength of association between VACs and the verbs that fill them (i.e., verb–VAC combinations) are developed. These indices are then compared against traditional indices of syntactic complexity (e.g., mean length of T-unit and mean length of clause) with regard to their ability to model one aspect of holistic scores of writing quality in Test of English as a Foreign Language (TOEFL) independent essays. Indices related to usage-based theories of syntactic development explained greater variance (R2 =.142) in holistic scores of writing quality than traditional methods of assessing syntactic complexity (R2 =.058). The results have important implications for modeling syntactic sophistication, L2 writing assessment, and AES systems.",
author = "Kristopher Kyle and Scott Crossley",
year = "2017",
month = "10",
day = "1",
doi = "10.1177/0265532217712554",
language = "English",
volume = "34",
pages = "513--535",
journal = "Language Testing",
issn = "0265-5322",
publisher = "SAGE Publications Ltd",
number = "4",

}

Assessing syntactic sophistication in L2 writing : A usage-based approach. / Kyle, Kristopher; Crossley, Scott.

In: Language Testing, Vol. 34, No. 4, 01.10.2017, p. 513-535.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Assessing syntactic sophistication in L2 writing

T2 - A usage-based approach

AU - Kyle, Kristopher

AU - Crossley, Scott

PY - 2017/10/1

Y1 - 2017/10/1

N2 - Over the past 45 years, the construct of syntactic sophistication has been assessed in L2 writing using what Bulté and Housen (2012) refer to as absolute complexity (Lu, 2011; Ortega, 2003; Wolfe-Quintero, Inagaki, & Kim, 1998). However, it has been argued that making inferences about learners based on absolute complexity indices (e.g., mean length of t-unit and mean length of clause) may be difficult, both from practical and theoretical perspectives (Norris & Ortega, 2009). Furthermore, indices of absolute complexity may not align with some prominent theories of language learning such as usage-based theories (e.g., Ellis, 2002a,b). This study introduces a corpus-based approach for measuring syntactic sophistication in L2 writing using a usage-based, frequency-driven perspective. Specifically, novel computational indices related to the frequency of verb argument constructions (VACs) and the strength of association between VACs and the verbs that fill them (i.e., verb–VAC combinations) are developed. These indices are then compared against traditional indices of syntactic complexity (e.g., mean length of T-unit and mean length of clause) with regard to their ability to model one aspect of holistic scores of writing quality in Test of English as a Foreign Language (TOEFL) independent essays. Indices related to usage-based theories of syntactic development explained greater variance (R2 =.142) in holistic scores of writing quality than traditional methods of assessing syntactic complexity (R2 =.058). The results have important implications for modeling syntactic sophistication, L2 writing assessment, and AES systems.

AB - Over the past 45 years, the construct of syntactic sophistication has been assessed in L2 writing using what Bulté and Housen (2012) refer to as absolute complexity (Lu, 2011; Ortega, 2003; Wolfe-Quintero, Inagaki, & Kim, 1998). However, it has been argued that making inferences about learners based on absolute complexity indices (e.g., mean length of t-unit and mean length of clause) may be difficult, both from practical and theoretical perspectives (Norris & Ortega, 2009). Furthermore, indices of absolute complexity may not align with some prominent theories of language learning such as usage-based theories (e.g., Ellis, 2002a,b). This study introduces a corpus-based approach for measuring syntactic sophistication in L2 writing using a usage-based, frequency-driven perspective. Specifically, novel computational indices related to the frequency of verb argument constructions (VACs) and the strength of association between VACs and the verbs that fill them (i.e., verb–VAC combinations) are developed. These indices are then compared against traditional indices of syntactic complexity (e.g., mean length of T-unit and mean length of clause) with regard to their ability to model one aspect of holistic scores of writing quality in Test of English as a Foreign Language (TOEFL) independent essays. Indices related to usage-based theories of syntactic development explained greater variance (R2 =.142) in holistic scores of writing quality than traditional methods of assessing syntactic complexity (R2 =.058). The results have important implications for modeling syntactic sophistication, L2 writing assessment, and AES systems.

UR - http://www.scopus.com/inward/record.url?scp=85029679808&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029679808&partnerID=8YFLogxK

U2 - 10.1177/0265532217712554

DO - 10.1177/0265532217712554

M3 - Article

AN - SCOPUS:85029679808

VL - 34

SP - 513

EP - 535

JO - Language Testing

JF - Language Testing

SN - 0265-5322

IS - 4

ER -