This study was conducted in order to develop a computerized test to measure the level of mathematic achievement and related cognitive functions in children and adolescents in South Korea.
The computerized Comprehensive Learning Test-Mathematic (CLT-M) consists of the whole number computation test, enumeration of dot group test, number line estimation test, numeral comparing test (magnitude/distance), rapid automatized naming test, digit span test, and working memory test. To obtain the necessary data and to investigate the reliability and validity of this test, 399 children and adolescents from kindergarten to middle school were recruited.
The internal consistency reliability of the CLT-M was high (Cronbach’s alpha=0.76). Four factors explained 66.4% of the cumulative variances. In addition, the data for all of the CLT-M subtests were obtained.
The computerized CLT-M can be used as a reliable and valid tool to evaluate the level of mathematical achievement and associated cognitive functions in Korean children and adolescents. This test can also be helpful to detect mathematical learning disabilities, including specific learning disorder with impairment in mathematics, in Korea.
Learning disorder (LD) is a condition that refers to a heterogeneous group of disabilities manifested by significant difficulties in the acquisition and use of listening, speaking, reading, writing, reasoning, or mathematical abilities. This condition is intrinsic to the individual and related to brain dysfunction. Although LD may occur concomitantly with other handicap conditions (sensory impairment, mental retardation, serious emotional disturbance), or with extrinsic influences (cultural differences, insufficient or inappropriate instructions), it is not the direct result of these conditions or influences.^{1)} In addition, the newer term ‘specific learning disorder (SLD)’ in the Diagnostic and Statistical Manual of Mental Disorders, 5^{th} edition means a disorder with difficulties to listen, think, speak, read, write, spell, or perform mathematical calculations due to one or more dysfunctions of the related neuropsychological processes.^{2)} This term does not include learning problems that are primarily resulted from visual, hearing, or motor disabilities, mental retardation, emotional disturbance, or environmental, cultural, or economic factors according to the Individual with Disabilities Education Improvement Act of 2004.^{3)} According to previous primitive surveys in Korea, the prevalence rate of SLD in school-age children is ranged 1.2-1.5%,^{4},^{5)} and the prevalence rate of dyscalculia is ranged 5-10%, which is similar to dyslexia.^{6)} However, to date, no systematic study diagnosed by a reliable tool has been conducted.
The consequences of dyscalculia are not less severe than those of dyslexia, even though there has been insufficient dyscalculia researches.^{7)} A large cohort study found that low numeracy was more of a handicap for an individual’s life chances than low literacy. People with dyscalculia depended on more assistance in school, and earned less. They also had higher risk of depression, poor physical health and illegal problems.^{6)}
For effective management of SLD with impairment in mathematics, prior to intervention, a valid standardized diagnostic test to obtain an accurate status quo of the children and adolescents with dyscalculia is necessary. Most previously published assessments of mathematical difficulties evaluated performances on both standardized mathematical achievement and measurement of underlying cognitive abilities.^{8)} It is because understanding of dyscalculia has been based on the concept from the classical diagnostic criteria according to the Individual with Disabilities Education Improvement Act of 1990, that is, “low achievement on standardized tests compared to expected levels of achievement based on underlying ability, age, and educational experience.”^{9)} Until now, the Neuropsychological Test Battery for Number Processing and Calculation in Children,^{10)} the Mathematical Abilitiesthird edition, the Wide Range Achievement Test 4, the Wechsler Individual Achievement Test-third edition,^{11)} the Mathematics Competency, the Dyscalculia Screener, the DyscalculiUM have been developed and widely used to evaluate dyscalculia.^{8)} In particular, the Dyscalculia Screener and the DyscalculiUM were designed as the computerized tests to assess more accurate, sensitive, and objective cognitive processing speed relative to paper-and-pencil tests. While the Dyscalculia Screener is for children aged 6-14 years, developed to identify dyscalculia by measuring response accuracy and response time, the DyscalculiUM is the first webbased solution for screening for dyscalculia in adults and learners in post-16 education, designed to screen large groups of students and individuals and takes less than an hour to complete. However, the Dyscalculia Screener cannot differentiate the subtypes of calculation problems.^{12)}
To the best of our knowledge, there has not yet been a computerized assessment tool developed for Korean children and adolescents with mathematical capabilities and underlying cognitive functions. Thus this study aimed to develop the computerized Comprehensive Learning Test-Mathematics (CLTM) to evaluate the basic numeric abilities as well as the related to the cognitive processes, which can help to identify the subtypes of calculation difficulties in children and adolescents in Korea.
Authors developed 8 subtests based on prior studies that had conducted to develope the objective tests to detect mathematical problems and underlying neurocognitive problems. According to the careful reviews, we determined the composition of the CLT-M, as follows. The CLT-M consists of 1 subtest to evaluate mathematical achievement including both accuracy and fluency, and 7 subtests to evaluate cognitive processing related to mathematics. It takes approximately 50 minutes to complete the entire test.
Whole number computation test^{13)}
This test aims to measure accuracy and speed of computation. Examinee is instructed to calculate an arithmetic problem presented on the computer screen as quickly as possible during a given period of time, and touch the correct answer on the screen.
Numeric comparing test (magnitude)^{14)}
This test is to select the higher number between two different numbers as quickly as possible.
Numeric comparing test (distance)^{15)}
The test is to compare numerical distances from a reference number to two different numbers. Examinee has to select the closer number from a reference number as quickly as possible.
Enumeration dot group test^{16)}
The test is for evaluating the ability to count the number of dots as quickly as possible. Examinee is instructed to count the number of dots shown on screen.
Number line estimation test^{17)}
This test is to evaluate the ability to point a number on the horizontal number line. Examinee should estimate the relative position of the number shown on screen, and then indicate the position on the horizontal line.
Rapid automatized naming test (object)^{18)}
This test aims to measure information processing speed. Examinee is instructed to name the objects shown on screen as quickly as possible.
Spatial working memory test^{19)}
The test is to measure spatial short-term memory and working memory. Recall the order of the blocks that get marked, and then in the reverse orders.
Digit span test^{20)}
The test aims to evaluate verbal short-term memory and working memory. After listening and memorizing a series of numbers, touch the numbers in forward and backward sequence, respectively.
Constitution of the CLT-M is shown in Table 1.
Test-retest reliability test was conducted to 20 children and adolescents in two-week intervals, using the paired t-test and the Pearson correlation analysis.
To test construct validity, the principal components factor analysis with oblique rotation was conducted.
From December 2013 to January 2014, 399 children and adolescents participated in the study. They were 5-14 years old, from the last year of kindergarten to middle school, dwelling in Seoul and Gyeonggi Province, South Korea. Subjects with mental retardation, sensory impairment, serious mental and neurological diseases and subjects who were incapable of performing the test for any other reasons were excluded, based on interviews with subjects and their caretakers. If available, caretakers also considered the results of the short version of intelligence tests that had applied at school previously. The average age was 11.63 (SD=2.69) years. The grade and gender distribution of the study subjects is presented in Table 2. This study protocol was approved by the Institutional Review Board of the Konkuk University, and informed consent and ascent was obtained from caretakers and participants, respectively.
All of the analyses were performed using SPSS 22.0 for Windows (IBM Corp., Armonk, NY, USA). Descriptive statistics and principal components factor analysis were conducted. The cut-off for statistical significance was set at p<0.05 two-tailed="" p="">
As a result of the paired t-test, there was no significant difference in all subtests, and mean Pearson correlation efficient was 0.87 (Table 3).
As a result of the factor analysis conducted to investigate the construct validity, there were four factors that could explain 66.35% of the cumulative variances. Factor 1, which explained 44.93% of the total variance, was a speed factor that included the whole number computation test, mean response time of the numeral comparing test (magnitude), mean response time of the numeral comparing test (distance), the enumeration of dot group test, the rapid naming test, and the working memory test. Factor 2, which explained 10.65% of the total variance, was an auditory working memory factor that included backward correct response and backward span of the digit span test. Factor 3, which explained 7.16% of the total variance, was an auditory simple attention factor that included forward correct response and forward span of the digit span test. Factor 4, which explained 3.61% of total variance, was an accuracy factor that included correct response rate of the numeral comparing test (magnitude), correct response rate of the numeral comparing test (distance), and correct response rate of the number-line estimation test (Table 4), and mean error rate of the number line estimation test.
The normative data of the CLT-M in Korean children and adolescents were obtained and are presented in Table 5 and 6.
As the result of this study, we developed the computerized test to evaluate the computational accuracy and fluency as well as the related underlying cognitive functions in Korean children and adolescents. Thus we will be able to detect children and adolescents with mathematical learning disabilities more accurately and determine the effectiveness of educational interventions more objectively. In addition, early and age-appropriate interventions for mathematical disability can be more feasible and the quality of life of impaired people may be improved.
For correct and quick calculation, highly complex brain activities for diverse cognitive processes are required. Various cognitive functions such as visuospatial construction ability, working memory including attention control, reasoning, and verbal capability as well as fundamental numeracy should be utilized.^{21)} Numerical processing capacity of the parietal lobe are used as a domain-specific function^{22}-^{24)} and central executive function and working memory capacity of the frontal lobe are used as domain-general function.^{25},^{26)} It has been suggested, in various cases of mathematical learning disabilities, that if numerical processing of the parietal lobe is selectively impaired, pure developmental SLD with impairment in mathematics can be developed, and if malfunctions exist in other brain areas where are responsible for general cognitive functions, more complex kinds of SLD with impairment in mathematics such as SLD with impairment in mathematics accompanied with impairment in reading or attention deficit can be rendered.^{10,27},^{28)} Although the CLTM originally aimed to assess the domain-specific functions in order to diagnose dyscalculia, the tasks related to measure domain-general cognitive functions like working memory were also included. Because working memory capability is important to compensate dyscalculic handicap^{29)} as well as patients with SLD with impairment in mathematics plus comorbid conditions such as attention deficit,^{30)} it is useful to have information on working memory for helping people with SLD with impairment in mathematics.
One of the core skills which are necessary to perform mathematical calculation is a numeric computation ability that is related to understanding the fundamental operations of arithmetic. The whole number computation is basic for more complex forms of math skills such as decimal and fraction calculations. According to previous researches, children and adolescents with SLD with impairment in mathematics had lower computation fluency^{31)} and more procedural bugs and slips.^{32)} This was because they lacked awareness of the skills, strategies, and resources which were needed to perform operation tasks, and often failed to apply appropriate strategies to solve math problems.^{33)} They also had difficulties in retrieving math facts from long-term memory even after intensive practices,^{34)} and lacked visual monitor ability.^{35)} In this study, the whole number computation test was used to measure computation performance ability including the level of computing strategies as well as accuracy and fluency of computation, which can be used as crucial guideline for determining the details for effective intervention.
For doing arithmetic, understanding numeric concept and numeric symbolic system are essential. Prior studies suggested that the easiest way to reveal one’s numerical concept was to measure the ability of dealing with numbers by comparative judgment. In case of SLD with impairment in mathematics, the performance of comparative judgment is significantly slow and inaccurate. The numeric comparing/magnitude test of this study measures the ability to deal with quantitative concept and distance concept of numbers. In general, performance is modulated by the distance between stimuli, and response time declines as the distance between stimuli increases.^{36},^{37)} However, children with SLD with impairment in mathematics tend to have inconsistent distance effect and higher error rate than normal healthy children,^{14},^{38)} because dyscalculic children have no or poor mental number line estimation capability which can be the source of distance effect.^{22)}
Subitizing and counting abilities are also crucial for arithmetic. While subitizing is to perceive the number of a group of items at a glance without counting, counting means to indicate the numbers consequently up to a particular number. In general, one to four objects are considered to be in the subitizing range, and five to nine objects are considered to be in the counting range. Children with SLD with impairment in mathematics have difficulties in understanding diverse symbols of number, for example, ‘5’, ‘five’, or ‘IIIII’, and in rapidly connecting the cardinality to digit, owing to neuropsychological problems such as working memory deficit and dysfunctional connection between number sense and verbal equivalent. In previous studies, counting strategy has been suggested as the major predictor of mathematical learning disabilities, particularly in preschoolers to the first year of primary school children.^{39}-^{41)} A smaller subitizing range can be associated with genetic SLD with impairment in mathematics.^{42},^{43)} Therefore, evaluating subitzing and counting abilities related to number cognition is a proper method for SLD with impairment in mathematics diagnosis.^{24,44},^{45)} By the enumeration dot group test in this study, we can measure the subitizing and counting abilities of Korean children and adolescents.
Moyer and Landauer^{46)} suggested that people converted written or auditory numbers into analog magnitudes. The digits, which are representing external magnitude, automatically induce the internal array of magnitudes, known as the mental number line.^{47)} Normally inherent numeracy forms the mental number line through education in elementary school. Especially the vertical number line was regarded as the foundation of higher mathematical thinking.^{10)} Formation of the mental number line means that the distance between two continuous numbers on the mental number line is constant regardless of the location of them. For example, we know that the distance between ‘2’ and ‘3’ is same as the distance between ‘98’ and ‘99’. At the same time, the higher number, the more it is compressed on the mental number line.^{48},^{49)} It is perceived as being located in a vertical line, depending on the natural logarithm scale, and the perceived distance between ‘100’ and ‘101’ appears shorter than the perceived distance between ‘2’ and ‘3’. However, dyscalculic children, who have a problem with number-quantity representation on the mental number line, do not follow a natural log model, rather they have the more compressed distanceperception of the smaller number.^{42)} In order to evaluate the spatial and temporal aspects of number sense, the estimation test was included in this study.
It has been reported that a deficit in information processing efficiency is one of the important cognitive characteristics of learning disabilities.^{50)} Rapid automatized naming (RAN) has been widely used to identify reading disabilities and has recently been turned out a useful correlating factor and predictor of mathematical learning disabilities^{40,51}-^{53)} because RAN measures not only visual-verbal connection but also retrieval speed of phonological information from long-term memory. According to previous researches, relative to typically achieving children, English-speaking children with SLD with impairment in mathematics were slower on the number subtest and more unstable on the letter subtest in the RAN.^{43,54},^{55)} However, Korean-speaking children with SLD with impairment in mathematics were slower on the object subtest.^{51}) Therefore, in order to evaluate information processing efficiency of Korean children, the object RAN test was included in this study.
Limitations of this study are as follows. First, the national representativeness of the sample is somewhat lacking because children and adolescents were recruited only in Seoul and Gyeonggi Province. In order to compensate for this, it is necessary to add supplement groups residing in rural areas and other provinces of Korea. Second, there are no data on the clinical effectiveness. Therefore, it is necessary to verify the clinical validity by the studies with patients with SLD with impairment in mathematics. Third, for the selection of study subjects, a confirmatory intelligence test was not applied. For more precise screening, it needs to be done, because SLD differs from general learning difficulties associated with intellectual disability, and usually occurs in the presence of normal levels of intellectual functioning.
Nevertheless, the CLT-M is the only available assessment tool to evaluate the mathematical capability and underling cognitive processes in Korea. Thus, it can be a useful tool to diagnose SLD with impairment in mathematics, if it is used with any standardized measurements of intelligence. Because typical SLD with impairment in mathematics including dyscalculia has no intelligence impairment. In addition, this computerized test can be applied more easily and accurately, particularly in the aspect of reaction time, in comparison with pencil-and-paper tests.