search for




 

Reliability and Validity of the Korean Translation of Quantitative Checklist for Autism in Toddlers: A Preliminary Study
J Korean Acad Child Adolesc Psychiatry 2018; 29(2): 80-85
Published online April 1, 2018
© 2018 Korean Academy of Child and Adolescent Psychiatry.

Subin Park1, Eun-Kyung Won2, Ji Hyun Lee3, Soyoung Yoon3, Eun Jin Park3,4, and and Yeni Kim2

1Department of Research Planning, Mental Health Research Institute, National Center for Mental Health, Seoul, Korea,
2Department of Child and Adolescent Psychiatry, National Center for Mental Health, Seoul, Korea,
3Goyang Child & Adolescent Mental Health Welfare Center, Goyang, Korea,
4Department of Psychiatry, Inje University Ilsan Paik Hospital, Goyang, Korea
Correspondence to: Yeni Kim, Department of Child and Adolescent Psychiatry, National Center for Mental Health, 127 Yongmasan-ro, Gwangjin-gu, Seoul 04933, Korea Tel: +82-2-2204-0321, Fax: +82-2-2204-0355, E-mail: yenikim@korea.kr
Eun-Jin Park, Department of Psychiatry, Inje University Ilsan Paik Hospital, 170 Juhwa-ro, Ilsanseo-gu, Goyang 10380, KoreaTel: +82-31-910-7544, Fax: +82-31-908-9221, E-mail: uma99@daum.net
Received January 2, 2018; Accepted January 18, 2018.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract

Objectives:

We aimed to assess the test-retest reliability, internal consistency, and validity of the Korean version of the Quantitative Checklist for Autism in Toddlers (Q-CHAT).

Methods:

The Korean version of the Q-CHAT and the Korean version of the Child Behavior Checklist (CBCL) 1.5-5 were completed by parents of 24 toddlers and preschoolers with autism spectrum disorder (ASD) and 80 unselected toddlers and preschoolers. Parents of the ASD group also completed the Social Communication Questionnaire (SCQ), and Childhood Autism Rating Scale (CARS) scores were obtained from medical records.

Results:

The ASD group scored higher on the Q-CHAT than the unselected group. The Cronbach’s alpha coefficient of the Q-CHAT was 0.658, and test-retest reliability was calculated to be 0.836. The estimated area under the curve was 0.793. The total scores of the Q-CHAT in the ASD group demonstrated significant positive correlations with findings regarding pervasive development problems in the CBCL, SCQ, and CARS. A total score of 33.5 may be a useful cutoff point to use when identifying toddlers at risk of ASD.

Conclusion:

The Korean version of the Q-CHAT has good reliability and validity and can be used as a screening tool in order to identify toddlers and preschool children at risk of ASD.

Keywords : Autism, Checklist, Toddler, Quantitative Checklist for Autism in Toddlers, Validation
INTRODUCTION

Autism is a developmental disorder characterized by persistent impairment in social communication, as well as repetitive and restricted patterns of behaviors or interests. Autism was previously classified as a diagnostic subcategory of pervasive developmental disorders (PDD). In the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), published by the American Psychiatric Association, the PDD label was removed and replaced with the term autism spectrum disorder (ASD).1) This label takes into account the fact that patients with ASD have varying symptoms, ranging from mild to severe, and therefore recognizes it as a disorder on a continuous spectrum.

Although early detection, screening, and interventions at an appropriate period of ASD are crucial, a diagnosis is often delayed due to various reasons, including insufficient standardized screening tools and treatment resources, as well as parental lack of awareness. The Korean Ages and Stages Questionnaires (K-ASQ) is being used as a developmental screening test in the present study, as part of the National Health Screening Program for Infants and Children, which has been enforced since 2010, following a revision in 2009. The K-ASQ is an assessment instrument developed by translating the original questionnaire (developed in the USA) into Korean and making appropriate revisions. Its diagnostic value has been tested (reliability 0.75-0.86). However, it is suggested to be weak in measuring social and emotional development.2)

A Korean version of the Social Communication Questionnaire (SCQ) is a screening test for ASD in children aged 4 and older.3) The SCQ is a 40-item parent report questionnaire with items that were extracted from the Autism Diagnostic Interview-Revised (ADI-R), an ASD diagnostic tool. Thus, the SCQ is effective as an initial screening test for ASD. Parents assess their children on three subscales of the SCQ: social interactions; language and communication; and restricted, repetitive, and stereotypic patterns of behavior.2)

Screening measures used in the UK and USA include the Modified Checklist for Autism in Toddlers (M-CHAT) and the Quantitative Checklist for Autism in Toddlers (Q-CHAT).4) The CHAT was developed based on behaviors that children with typical development show around 18 months of age. It was the first screening tool developed, and it is based on assessments from parents and professionals.5) The M-CHAT is a parent report measure that includes sensory abnormalities and repetitive behaviors in addition to the original items of CHAT. The M-CHAT, however, has low sensitivity, with strict wording and yes/no format. In order to resolve this issue, Q-CHAT was developed by the Autism Research Center (ARC) of Cambridge University in 2008.

The Q-CHAT is a screening tool that quantitatively rates the frequency of characteristic ASD behaviors, including sensory issues, behavior problems, and social skills, through parent report.6) In the Q-CHAT, the items of joint attention and pretend play are retained from the M-CHAT, in addition to items based on DSM-IV and International Classification of Diseases-10 diagnostic criteria. Another alteration in the Q-CHAT is a shift from the yes/no format to responses that can be quantified, given that some children who are at high risk of ASD show key behaviors of ASD in low frequencies. The Q-CHAT is quick to administer; it takes approximately 5-10 minutes for parents to complete the questionnaire. The aims of our study were to develop a Korean version of the Q-CHAT as a screening instrument for early ASD detection and to test its reliability and validity in autistic and non-autistic samples.

METHODS

Participants and procedures

The study was conducted from May 1, 2014 to October 31, 2014. Toddlers and preschool children aged 2-5 with a diagnosis of ASD were selected for the study. Of the 38 children who passed the initial screening, 24 children were diagnosed with ASD by a child psychiatrist and participated in the study. The rest (n=14) were diagnosed with developmental delays (n=8) and language disorder (n=6), and were excluded from the study. The Q-CHAT, SCQ, and Child Behavior Checklist (CBCL) 1.5-5 were completed by the parents of our participants. The Childhood Autism Rating Scale (CARS) scores were obtained from the children’s medical records. The parents of the ASD group completed a second Q-CHAT 4 weeks later. In addition, the Q-CHAT and CBCL 1.5-5 were completed by parents of 80 toddlers and preschool children aged 2-5 years who were recruited from community childcare centers that were participating in the Infants and Toddlers Early Developmental Screening program of the Goyang City’s Child and Adolescent Mental Health Center (CAMHC). The research information and documents were sent to six childcare centers and kindergartens that were part of the Developmental Screening program of the CAMHC. Four institutions agreed to participate in the study. The purposes and procedures of the study were explained to the parents, and those who agreed to participate were included in the study. This study was approved by the Institutional Review Board of the National Center for Mental Health (2014-17).

Assessment instruments

Quantitative Checklist for Autism in Toddlers

The translation of Q-CHAT was approved by the developer, Baron-Cohen, and the ARC of Cambridge University. The Q-CHAT was first translated from original language (English) to the target language (Korean) by two psychiatrists, and back-translated into English by a psychiatrist who was equally fluent in Korean and English. The two translations were revised to obtain a final Korean version of the Q-CHAT. The Q-CHAT is a 25-item parent report screening instrument. Half of the items are reverse scored. Each item is scored on a scale ranging from 0 to 4 according to the frequency of symptoms. A higher total Q-CHAT score indicates the presence of more ASD traits. An ambiguous answer or a lack of response was given a score of 0.6)

Social Communication Questionnaire

The SCQ is a screening instrument developed by extracting key questions on deviant characteristics from normal development from the ADI-R. It is a 40-item, parent report questionnaire. The parents assess their children on 3 subscales: social interactions, language and communication, and restricted, repetitive, and stereotypic behaviors. The Korean version of the SCQ was standardized by Kim et al.7)

Child Behavior Checklist

The CBCL was developed by Achenbach and Rescorla,8) and the Korean version was standardized by Oh et al.9) The CBCL 1.5-5 consists of 2 domains: a problem behavior scale and DSM-oriented scale. The problem behavior scale includes emotionally reactive, anxious/depressed, somatic complaints, withdraw, attention problems, aggressive behavior, sleep problems, other behavioral problems, internalizing problems total score, externalizing problems total score, and total problems score. The DSM-oriented scale includes affective problems, anxiety problems, pervasive developmental problems, attention-deficit hyperactivity problems, and oppositional defiant problems. In a standardization study, the Cronbach’s alpha reliability coefficient for the total score of problem behaviors scale was 0.94 and that of the DSM-oriented scale was 0.56-0.89.

Childhood Autism Rating Scale

The CARS is a diagnostic instrument developed to distinguish autism from other developmental disorders and assess the severity of symptoms of ASD. It is a quantitative and objective measure based on direct behavior observation rather than subjective clinical assessment. The CARS may be used in all age groups, including preschool children. It consists of 15 domains (relationships to people, nonverbal communication, verbal communication, etc.) scored on a scale ranging from 1 (normal for age) to 4 (severely abnormal). The sum of the scores of the 15 domains is used for the diagnostic categorization.10)

Statistical analysis

Descriptive statistics were used to analyze the sex, age, and scores of each instrument for the unselected and ASD groups. The Cronbach’s alpha coefficient was calculated to assess the internal consistency of the assessment instruments. The test-retest reliability was examined through a second Q-CHAT obtained approximately 4 weeks after the first test for the ASD group. The Receiver Operating Characteristic (ROC) curve was used to determine the optimal cutoff score. In order to determine the criterion validity of the Korean version of the Q-CHAT, correlations between the total scores of the Q-CHAT and the total scores of CARS, CBCL 1.5-5, and SCQ were evaluated using Pearson’s r coefficient. SPSS 21.0 (IBM Corp., Armonk, NY, USA) was used for all statistical analyses. Considering multiple comparisons, the statistical significance was adjusted downward to less than 0.01 (=0.05/5 subscale of CBCL 1.5-5).

RESULTS

Twenty-four toddlers and preschool children in the ASD group (16 male, 8 female) were between 2 and 5 years of age, with a mean age of 47.7 months. Eighty toddlers and preschoolers in the unselected group (42 male, 38 female) were between 2 and 4 years of age, and the mean age was 46.2 months. The mean Q-CHAT scores of the ASD group and the unselected group were 39.1±9.2 and 29.7±7.5, respectively (Table 1). The Shapiro-Wilk test for normality of data showed that the Q-CHAT scores followed a normal distribution. The Q-CHAT score distributions of the two groups were presented in Fig. 1. There were no significant sex differences in the total Q-CHAT scores (39.6±10.3 for male and 38.3±6.9 for female, p=0.749) in the ASD group, whereas the total Q-CHAT scores of boys were significantly higher than girls in the unselected group (32.8±7.4 for male and 26.3±6.1 for female, p<0.001). The total Q-CHAT scores and age were negatively correlated (in the ASD group, r=-0.315; in the unselected group, r=-0.239).

Demographic characteristics of the study populations

ASD group (n=24) Typical group (n=80)  x2/ p-value Total (n=104)
Sex, n (%)1.500.220
 Male16 (66.7)42 (52.5)58 (55.8)
 Female8 (33.3)38 (47.5)46 (44.2)
Age (months, mean±SD)47.71±13.1446.18± 8.570.540.59446.53±9.76
QCHAT (mean±SD)39.13±9.1929.70±7.505.12<0.00131.87±8.83
CBCL 1.5-5 (mean±SD)
 Affective problem59.17±8.8953.70±6.272.760.01054.92±7.26
 Anxiety problem55.87±5.6853.64± 5.971.600.11354.14±5.95
 Attention-deficit hyperactivity problem62.30±7.6654.89±6.704.53<0.00156.54± 7.55
 Oppositional defiant problem72.83±6.7754.49±7.373.91<0.00156.05±8.06
 Pervasive developmental problem61.48±8.1954.25±7.4310.77<0.00158.40±10.63
CARS (mean±SD)29.17±7.00
SCQ (mean±SD)18.71±5.87

ASD: autism spectrum disorder, CARS: Childhood Autism Rating Scale, CBCL: Child Behavior Checklist, SCQ: Social Communication Questionnaire, SD: standard deviation, Q-CHAT: Quantitative Checklist for Autism in Toddlers


Fig. 1.

Comparison for the score distributions for the typical group (n=80) and the ASD group (n=24). ASD: autism spectrum disorder.


The Cronbach’s alpha coefficient of the Korean version of the Q-CHAT was 0.681 (n=104), showing an adequate internal consistency. The Cronbach alpha coefficient increased to 0.711 when item 3 (i.e., “When your child is playing alone, does s/he line objects up?”), and item 18 (i.e., “Does your child echo things s/he hears?”) were removed (Table 2).

Correlations between each item and the total score (n=104)

  Item Item-total correlation Cronbach alpha if them deleted
1. Look when call name0.4950.632
2. Eye contact0.5610.629
3. Line objects up*-0.2920.693
4. Understand child’s speech0.6070.605
5. Protoimperative pointing0.1020.660
6. Protodeclarative pointing0.3460.635
7. Interest maintained by spinning object*-0.0660.676
8. Number of words*0.5270.620
9. Pretend play0.5030.617
10. Follow a look0.5010.620
11. Sniff/lick unusual objects*0.1630.656
12. Use of hand as tool*0.0150.675
13. Walk on tiptoes*0.0820.660
14. Adapt to change in routine0.2520.650
15. Offer comfort0.4910.618
16. Do same thing over and over again*0.2440.646
17. Typicality of first words0.3670.635
18. Echolalia*-0.3870.705
19. Gestures0.2550.646
20. Unusual finger movements*0.3920.632
21. Check reaction0.0530.663
22. Maintenance of interest*-0.0590.673
23. Twiddle objects repetitively*0.1980.651
24. Oversensitive to noise*0.1930.651
25. Stare at nothing with no purpose*0.3500.639

*indicates item is reverse scored. SD: standard deviation


In order to assess test-retest reliability, the parents were invited to complete the questionnaire a second time an average of 4 weeks later. Of those invited parents, 83.3% (n=20) responded. The Pearson’s r coefficient of the test-retest reliability was 0.836 (p<0.001), showing high reliability.

The ROC curve analysis of scores showed an optimal cut-off score of 33.5, and the sensitivity and specificity were 0.75 and 0.73, respectively. The estimated area under the curve was 0.793 (95% CI=0.683-0.903) (Fig. 2).

Fig. 2.

The ROC. ROC: Receiver Operating Characteristic.


For the ASD group, total Q-CHAT scores demonstrated significant positive correlations with the pervasive development problem score of the CBCL (r=0.712, p<0.001), SCQ (r=0.669, p<0.001), and CARS (r=0.550, p=0.007). However, total Q-CHAT scores were not significantly correlated with the affective problem, anxiety problem, attention-deficit hyper-activity problem, or oppositional defiant problem scores. In contrast, total Q-CHAT scores of the unselected group demonstrated significant positive correlations with the pervasive development problem score, as well as with the affective problem, anxiety problem, attention-deficit hyperactivity problem, and oppositional defiant problem scores (Table 3).

Correlations of CBCL, CARS, SCQ and Q-CHAT retest scores with Q-CHAT total scores

      Q-CHAT

ASD group (n=24) Typical group (n=80)
CBCL 1.5-5
Affective problem0.3580.360*
Anxiety problem0.465 (p =0.026)0.349*
Attention-deficit hyperactivity problem 0.486 (p=0.019)0.299*
Oppositional defiant problem0.2160.564**
Pervasive developmental problem0.712**0.474**
CARS0.550**
SCQ0.669**

*p<0.01,

**p<0.001.

ASD: autism spectrum disorder, CARS: Childhood Autism Rating Scale, CBCL: Child Behavior Checklist, SCQ: Social Communication Questionnaire, Q-CHAT: Quantitative Checklist for Autism in Toddlers


DISCUSSION

The aims of this study were to translate the Q-CHAT developed by the ARC of Cambridge University into Korean and to assess its resulting reliability and validity. The results of our study showed high correlations between the Korean version of the Q-CHAT and other currently available screening and diagnostic tools for ASD. The results also showed adequate internal consistency of the Korean version of the Q-CHAT. The ROC analysis showed good screening accuracy, and the test-retest reliability analysis exhibited a high correlation.

The mean age of the ASD group was 47.7 months, which was similar to the mean age (44.5 months) from a previous study.6) In the ASD group, the mean score of the Korean version of the Q-CHAT was 39.1, which was lower than the mean score (51.8) from a previous study.6) This could be partly due to a bias resulting from the small sample size of the ASD group in our study. In contrast, the mean score of the Korean version of the Q-CHAT was 29.7 in the unselected group, which was similar to the mean score in a previous study (26.7). The total Q-CHAT score tended to decrease with increasing age, which is consistent with the previous findings.6)

The Cronbach’s alpha coefficient for the internal consistency of the Korean version of the Q-CHAT was 0.651 (n=104). In a previous study evaluating internal consistency, the Cronbach’s alpha coefficient of the unselected group (n=754) was 0.67, which was similar to our finding, whereas the Cronbach’s alpha of the ASD group (n=160) in the previous study was 0.83, which was higher than that in the present study.6) Although the internal consistency of the Korean version of the Q-CHAT was adequate with a Cronbach’s alpha of 0.65, removing items 3 and 18 from the questionnaire may be considered in order to achieve a greater reliability.

The correlation coefficient of the test-retest reliability analysis was greater than 0.8. In general, values greater than 0.6 are considered to be acceptable levels of test-retest reliability. Therefore, the results of our study show that the Korean version of the Q-CHAT has good test-retest reliability. However, it is necessary to reexamine the results with a greater sample size in future studies.

The total Q-CHAT scores of the ASD group showed significant positive correlations with the pervasive developmental problem scores of the CARS, SCQ, and CBCL. The total Q-CHAT scores, however, did not show significant correlations with other subscales of the CBCL. As such, the Korean version of the Q-CHAT showed good discriminant validity. In contrast, the total Q-CHAT scores of the unselected group showed significant positive correlations with not only the pervasive developmental problem scores of the CBCL, but also with all other subscales (affective, anxiety, attention-deficit hyperactivity, oppositional defiant). This suggests that the Q-CHAT is not effective in distinguishing pervasive developmental problems from other affective and behavioral problems in children with typical development, but it is, however, in children with ASD. Thus, the Q-CHAT may be a useful screening tool in differentiating the children with ASD from the children with typical development.

A limitation of this study was the small sample size of the ASD group (n=24). As there were an insufficient number of toddlers and preschoolers who met diagnostic criteria, the mean score of the Q-CHAT was lower than the mean in a previous study. Additionally, this small sample size resulted in an acceptable rather than high level of internal consistency. Nonetheless, our study has a unique strength, as it is the first study to compare clinical variables of patients against the Q-CHAT, in contrast to previous studies, which failed to determine its validity in comparison with other available assessment instruments, due to lack of information about clinical variables such as psychological test scores.

CONCLUSION

The Korean version of the Q-CHAT has good reliability and validity and can be used as a screening tool in order to identify toddlers and preschool children at risk of ASD. Further studies with larger sample sizes for the ASD group and control group are needed to confirm our results.

Acknowledgments

This study was supported by Ministry of Health and Welfare (grant number: HI14C1098) and the intramural grant (No 2014-06) from the National Center for Mental Health. The authors would like to thank Eo-Son Sohm for her help with the initial arrangements for the translation.

Conflicts of Interest

The authors have no financial conflicts of interest.

References
  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5). Washington, DC: American Psychiatric Association Publishing; 2013.
  2. Moon JS. Review of National Health Screening Program for Infant and Children in Korea. J Korean Med Assoc 2010;53:377-385.
    CrossRef
  3. Yoo HJ. Korean version of Social Communication Questionnaire. Seoul: Hakjisa; 2008.
  4. Robins DL, Fein D, Barton ML, and Green JA. The Modified Checklist for Autism in Toddlers:an initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord 2001;31:131-144.
    Pubmed CrossRef
  5. Baron-Cohen S, Allen J, and Gillberg C. Can autism be detected at 18 months? The needle, the haystack, and the CHAT. Br J Psychiatry 1992;161:839-843.
    Pubmed CrossRef
  6. Allison C, Baron-Cohen S, Wheelwright S, Charman T, Richler J, and Pasco G et al. The Q-CHAT (Quantitative CHecklist for Autism in Toddlers):a normally distributed quantitative measure of autistic traits at 18-24 months of age:preliminary report. J Autism Dev Disord 2008;38:1414-1425.
    Pubmed CrossRef
  7. Kim JH, Sunwoo HJ, Park SB, Noh DH, Jung YK, and Cho IH et al. A validation study of the Korean version of Social Communication Questionnaire. Korean Acad Child Adolesc Psychiatry 2015;26:197-208.
    CrossRef
  8. Achenbach TM, and Rescorla LA. Manual for the ASEBA preschool forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2000.
  9. Oh KJ, Lee H, Hong KE, and Ha EH. Korean version of Child Behavior Checklist (K-CBCL) 1997.
  10. Soh EH, and Cheong YS. A study of reliability and vality of the Childhood Autism Rating Scale-Korean version. Kor J Neuropsychiatr Assoc 1992;31:471-477.


October 2018, 29 (4)
Full Text(PDF) Free

Social Network Service
Services

Cited By Articles
  • CrossRef (0)

Funding Information
  • Ministry of Health and Welfare(Ministry of Health, Welfare and Family Affairs)
      10.13039/501100003625
      HI14C1098