A review of the use of script-based tracking in CALL research for data sharing: applications providing meaning aids

Abstract

Using script-based tracking to gain insights into the way students learn or process language information can be traced as far back as to the 1980s. Nevertheless, researchers continue to face challenges in collecting and studying this type of data. The objective of this study is to propose data sharing through data repositories as a way to (a) ease the challenges that researchers face in collecting and studying this type of tracking data in non-CMC environments, (b) increase the use of this type of data to gain insights into second language learning, and (c) synthesize and enhance CALL research. To this end, the study outlines the functions of a potential data repository and reviews extant research studies. The review focuses on computer applications that used meaning aids to make the meaning of language items accessible during reading activities. The goal is to identify the required data fields for such repository by taking into account existing research studies. Finally, the study provides two potential research studies to exemplify how researchers can use the aggregated data from the repository to study research questions. Consequently, they may be able to draw stronger conclusions, thus enhancing the generalizability of CALL findings.

Keywords:

• computer logs
• data sharing
• gloss
• meaning aids
• secondary data
• tracking

Acknowledgements

The author would like to thank Ching-yeh Tzseng for conceptualizing and creating the process flowchart for the potential data repository. This study was partially supported by Charles Phelps Taft Research Center.

Notes

1. The author noted that during a presentation in a CALL conference, a researcher suggested that the meaningful behavioral data identified by the use of screen-capture software be examined to see if they could be captured by computer logs because it would greatly expedite research process.

2. In most of the studies reviewed, the learning was incidental, although one study investigated both incidental and intentional vocabulary learning (Peters, 2007).

3. Some examples of data repositories from various disciplines include: CHILDES, Geological Society of America (GSA) Data Repository, National Center for Ecological Analysis and Synthesis Data Repository, Open Language Archives Community, Stroke Imaging Repository Consortium (STIR), PSLC DataShop, and UC Irvine Machine Learning Repository.

4. The inferring ability was measured through a self-designed test (Hulstijn, 1993).

5. The analysis of time-on-task demonstrates that the low verbal ability subjects benefited from the online dictionary more than the high verbal ability subjects.

6. A data repository will have to comply with Institutional Review Board (IRB) rules. When learning data were first collected, the compliance with IRB rules would already have been ensured (e.g. data redacted to strip all individual identifiers to maintain privacy and confidentiality). Researchers who want to share such data will need to maintain compliance with IRB rules and consult IRB for any additional requirements on compliance. NIH web site on “Frequently Asked Questions. Data Sharing” has provided some general guidance on preparing data for sharing and archiving.

7. The data setup in Table 2 does not intend to be comprehensive. In other words, it does not include all the variables identified based on the review of studies.

8. If both visual and verbal information are provided in a single gloss, the data of such study is probably not usable for the second study suggested here.