Assessment of language in authentic Science inquiry reveals putative differences in epistemology

Melanie E. Peffer, Kristopher Kyle

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Science epistemology, or beliefs about what it means to do science and how science knowledge is generated, is an integral part of authentic science inquiry. Although the development of a sophisticated science epistemology is critical for attaining science literacy, epistemology remains an elusive construct to precisely and quantitatively evaluate. Previous work has suggested that analysis of student practices in science inquiry, such as their use of language, may be reflective of their underlying epistemologies. Here we describe the usage of a learning analytics tool, TAALES, and keyness analysis to analyze the concluding statements made by students at the end of a computer-based authentic science inquiry experience. Preliminary results indicate that linguistic analysis reveals differences in domain-general lexical sophistication and in domain-specific verb usage that are consistent with the expertise level of the participant. For example, experts tend to use more hedging language such as "may" and "support" during conclusions whereas novices use stronger language such as "cause." Using these differences, a simple, rulebased prediction algorithm with LOOCV achieved prediction accuracies of greater than 80%. These data underscore the potential for the use of learning analytics in simulated authentic inquiry to provide a novel and valuable method of assessing inquiry practices and related epistemologies.

Original languageEnglish
Title of host publicationLAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference
Subtitle of host publicationUnderstanding, Informing and Improving Learning with Data
PublisherAssociation for Computing Machinery
Pages138-142
Number of pages5
ISBN (Electronic)9781450348706
DOIs
Publication statusPublished - 2017 Mar 13
Event7th International Conference on Learning Analytics and Knowledge, LAK 2017 - Vancouver, Canada
Duration: 2017 Mar 132017 Mar 17

Publication series

NameACM International Conference Proceeding Series

Conference

Conference7th International Conference on Learning Analytics and Knowledge, LAK 2017
CountryCanada
CityVancouver
Period17/3/1317/3/17

Fingerprint

Students
Linguistics

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications

Cite this

Peffer, M. E., & Kyle, K. (2017). Assessment of language in authentic Science inquiry reveals putative differences in epistemology. In LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data (pp. 138-142). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3027385.3027425
Peffer, Melanie E. ; Kyle, Kristopher. / Assessment of language in authentic Science inquiry reveals putative differences in epistemology. LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data. Association for Computing Machinery, 2017. pp. 138-142 (ACM International Conference Proceeding Series).
@inproceedings{da49a9017096430ba9c2257f1237ab7c,
title = "Assessment of language in authentic Science inquiry reveals putative differences in epistemology",
abstract = "Science epistemology, or beliefs about what it means to do science and how science knowledge is generated, is an integral part of authentic science inquiry. Although the development of a sophisticated science epistemology is critical for attaining science literacy, epistemology remains an elusive construct to precisely and quantitatively evaluate. Previous work has suggested that analysis of student practices in science inquiry, such as their use of language, may be reflective of their underlying epistemologies. Here we describe the usage of a learning analytics tool, TAALES, and keyness analysis to analyze the concluding statements made by students at the end of a computer-based authentic science inquiry experience. Preliminary results indicate that linguistic analysis reveals differences in domain-general lexical sophistication and in domain-specific verb usage that are consistent with the expertise level of the participant. For example, experts tend to use more hedging language such as {"}may{"} and {"}support{"} during conclusions whereas novices use stronger language such as {"}cause.{"} Using these differences, a simple, rulebased prediction algorithm with LOOCV achieved prediction accuracies of greater than 80{\%}. These data underscore the potential for the use of learning analytics in simulated authentic inquiry to provide a novel and valuable method of assessing inquiry practices and related epistemologies.",
author = "Peffer, {Melanie E.} and Kristopher Kyle",
year = "2017",
month = "3",
day = "13",
doi = "10.1145/3027385.3027425",
language = "English",
series = "ACM International Conference Proceeding Series",
publisher = "Association for Computing Machinery",
pages = "138--142",
booktitle = "LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference",

}

Peffer, ME & Kyle, K 2017, Assessment of language in authentic Science inquiry reveals putative differences in epistemology. in LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 138-142, 7th International Conference on Learning Analytics and Knowledge, LAK 2017, Vancouver, Canada, 17/3/13. https://doi.org/10.1145/3027385.3027425

Assessment of language in authentic Science inquiry reveals putative differences in epistemology. / Peffer, Melanie E.; Kyle, Kristopher.

LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data. Association for Computing Machinery, 2017. p. 138-142 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Assessment of language in authentic Science inquiry reveals putative differences in epistemology

AU - Peffer, Melanie E.

AU - Kyle, Kristopher

PY - 2017/3/13

Y1 - 2017/3/13

N2 - Science epistemology, or beliefs about what it means to do science and how science knowledge is generated, is an integral part of authentic science inquiry. Although the development of a sophisticated science epistemology is critical for attaining science literacy, epistemology remains an elusive construct to precisely and quantitatively evaluate. Previous work has suggested that analysis of student practices in science inquiry, such as their use of language, may be reflective of their underlying epistemologies. Here we describe the usage of a learning analytics tool, TAALES, and keyness analysis to analyze the concluding statements made by students at the end of a computer-based authentic science inquiry experience. Preliminary results indicate that linguistic analysis reveals differences in domain-general lexical sophistication and in domain-specific verb usage that are consistent with the expertise level of the participant. For example, experts tend to use more hedging language such as "may" and "support" during conclusions whereas novices use stronger language such as "cause." Using these differences, a simple, rulebased prediction algorithm with LOOCV achieved prediction accuracies of greater than 80%. These data underscore the potential for the use of learning analytics in simulated authentic inquiry to provide a novel and valuable method of assessing inquiry practices and related epistemologies.

AB - Science epistemology, or beliefs about what it means to do science and how science knowledge is generated, is an integral part of authentic science inquiry. Although the development of a sophisticated science epistemology is critical for attaining science literacy, epistemology remains an elusive construct to precisely and quantitatively evaluate. Previous work has suggested that analysis of student practices in science inquiry, such as their use of language, may be reflective of their underlying epistemologies. Here we describe the usage of a learning analytics tool, TAALES, and keyness analysis to analyze the concluding statements made by students at the end of a computer-based authentic science inquiry experience. Preliminary results indicate that linguistic analysis reveals differences in domain-general lexical sophistication and in domain-specific verb usage that are consistent with the expertise level of the participant. For example, experts tend to use more hedging language such as "may" and "support" during conclusions whereas novices use stronger language such as "cause." Using these differences, a simple, rulebased prediction algorithm with LOOCV achieved prediction accuracies of greater than 80%. These data underscore the potential for the use of learning analytics in simulated authentic inquiry to provide a novel and valuable method of assessing inquiry practices and related epistemologies.

UR - http://www.scopus.com/inward/record.url?scp=85016489248&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85016489248&partnerID=8YFLogxK

U2 - 10.1145/3027385.3027425

DO - 10.1145/3027385.3027425

M3 - Conference contribution

AN - SCOPUS:85016489248

T3 - ACM International Conference Proceeding Series

SP - 138

EP - 142

BT - LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference

PB - Association for Computing Machinery

ER -

Peffer ME, Kyle K. Assessment of language in authentic Science inquiry reveals putative differences in epistemology. In LAK 2017 Conference Proceedings - 7th International Learning Analytics and Knowledge Conference: Understanding, Informing and Improving Learning with Data. Association for Computing Machinery. 2017. p. 138-142. (ACM International Conference Proceeding Series). https://doi.org/10.1145/3027385.3027425