A considerable number of children and adolescents visiting the department of child and adolescent psychiatry complain of attention problems. According to studies conducted in Korea, 5.9–8.5% of children and adolescents are known to have attention-deficit/hyperactivity disorder (ADHD) [1], and the number of patients diagnosed with ADHD is rapidly increasing [2]. ADHD is a neurodevelopmental disorder that occurs during childhood and adolescence, accompanied by inattention, impulsivity, and hyperactive behaviors. The main characteristics include problems in selective and sustained attention [3].

Attention in children and adolescents is critical as it is closely associated with academic performance, and many children and adolescents with ADHD tend to show a low level of academic performance. They find it difficult to complete home assignments and prepare for classes at school, and their inattentive and impulsive behaviors prevent them from achieving a high level of performance in examinations [4]. In a study on the parents of adolescents with ADHD, one of the most urgent problems for which they sought solutions was academic achievement, which implied that children and adolescents with ADHD as well as their parents regarded the subjective experience of academic problems as a highly serious matter in their daily lives [5]. Moreover, repeated low achievements or academic failure, when combined with hopelessness, low selfesteem, and negative stigma at school, often caused a secondary emotional problem [6]. Among a variety of emotional problems, anger, in particular, is known to be most frequently experienced by children and adolescents with ADHD. While anger is a natural feeling experienced in daily life, uncontrolled anger could cause physical or psychological problems. Numerous studies have reported that compared to normal children and adolescents, those with ADHD showed a high level of physical and verbal aggression with a markedly strong expression of anger [7,8]. Notably, as uncontrolled anger could potentially lead to various problems related to violent or defiant behaviors [9], care should be taken regarding secondary emotional problems alongside attention problems exhibited by children and adolescents with ADHD. Therapeutic intervention should be developed based on accurate assessments of ADHD.

However, problematic behaviors, including inattention and anger during childhood and adolescence, are also observed in various other psychiatric disorders such as depression, bipolar disorder, oppositional defiant disorder, and conduct disorder. In addition, the symptoms and therapeutic interventions may vary according to specific disorders or types. Therefore, it is very important to differentially diagnose ADHD and other disorders using accurate assessment tools. Thus, in actual clinical settings, assessments are made objectively and systematically through various methods regarding inattention, impulsivity, and anger control problems in children and adolescents, such as selfreporting questionnaires, rating scales for parents and guardians, clinical interviews, and psychological tests.

Among the tests for screening ADHD, the rating scale for parents and guardians is one of the most widely used tests as it is easy to perform and allows simple scoring and interpretation while reducing the possibility of under-rating that may occur in self-reporting questionnaires based on the ratings of children and adolescents with ADHD themselves due to inadequate self-introspection or evaluation abilities [10]. Moreover, it has been reported that for children and adolescents with ADHD, information from parents or guardians who are able to observe the child or adolescent every day is crucial; it is an essential tool for ADHD diagnosis and treatment together with the self-reporting questionnaires [11,12]. The rating scales for parents and guardians most commonly used in Korea today for assessing inattention, anger, and hyperactivity in children and adolescents with ADHD include the ADHD Rating Scale standardized in Korea [13] known as the Korean ADHD Rating Scale (K-ARS) [14], the Korean version of the IOWA Conners Rating Scale (K-IOWA) [15], and the Abbreviated Conners Rating Scale [16].

The DSM-5 Level 2 Cross-Cutting Symptom Measure–in-attention [Swanson, Nolan and Pelham, version IV (SNAP-IV)] and anger [Patient-Reported Outcome Measurement Information System (PROMIS) Anger] [1,17]. A study on pediatric subjects in the US reported high reliability for the scales [18], and the assessment tool is frequently used not only in clinical settings but also in clinical research. Shin et al. [19] translated SNAP-IV inattention and PROMIS Anger scales to Korean, and their content validity was verified in a preliminary study. In another study conducted in Korea, these scales were reported to be a reliable tool to measure psychological stress in children and adolescents [20]. In addition, as the SNAP-IV inattention scale and PROMIS Anger scale of the DSM-5 contain a small number of questions, they are expected to be useful in clinical settings for assessing the symptoms of inattention and anger in children and adolescents in a time-efficient manner and in the follow-up assessment of treatment effects and progress. The use of the scales has thus gradually increased among researchers in Korea [21]. Nevertheless, the reliability and validity of the scales are yet to be verified. Therefore, the present study investigated the reliability and validity of the Korean version of the DSM-5 Level 2 Cross-Cutting Symptom Measure–inattention and anger for parents and guardians of children aged 6–17 years, by targeting children and adolescents of various clinical groups with the aim of developing a reliable and valid standard outcome measure that can be used in ADHD screening and treatment evaluation in Korean clinical settings.

Subjects

We recruited children and adolescents aged 8–15 years and their parents who were visiting the department of child and adolescent psychiatry located in Seoul, Korea, during the period from January to December 2020. Based on the diagnosis of child psychiatrists using DSM-5 diagnostic criteria and the comprehensive psychological assessment of a licensed child clinical psychologist, the participants were classified as having ADHD, ADHD+anxious depression, depressive disorder, anxiety disorder, and tic disorder+somatic symptoms and were categorized into five groups accordingly. Only participants whose overall Korean Wechsler Intelligence Scale for Children, fourth edition scores were ≥IQ 70 and ≥Verbal Comprehension Index 80 were recruited in this study, and a total of 104 participants were included in the analysis. This study was conducted after permission was obtained from the Institutional Review Board of Seoul National University Hospital (IRB No. 1950-145-1030).

Assessment tools

SNAP is a scale developed by Swanson based on the ADHD diagnostic criteria of DSM-III [22], while its reconstruction based on the diagnostic criteria of DSM-IV led to SNAP-IV [23]. In this study, the Korean version of the scale, as translated by Shin et al. [19], was used. SNAP-IV is a rating scale for parents and guardians regarding attention-related difficulties observed in children and adolescents aged 6–17 years throughout classes, home assignments, reading, and playing during 2 weeks. It includes eight questions on a four-point Likert scale, with a total score ranging between 0 and 24. Higher scores indicate higher degrees of inattention. The mean total score is calculated by dividing the total or partial raw score by the number of questions (i.e., eight), while previous studies reported a satisfactory level of internal consistency at Cronbach’s α=0.84–0.95.

The DSM-5 Level 2 Cross-Cutting Symptom

PROMIS-SF was developed by the National Institutes of Health (NIH), and in this study, the Korean version of the PROMIS Emotional Distress, Anger, as translated by Shin et al. [19] was used. PROMIS is a rating scale for parents and guardians regarding the severity of emotions such as irritability, anger, or becoming enraged during 2 weeks. It includes five questions on a five-point Likert scale, with a total score ranging between 5 and 25. Higher scores indicate higher severity of anger. The raw scores of the five questions are added to produce the total raw score, and the respective T score is checked. A T score <55 is considered normal; T score ≥55 and <60 indicates a moderate level of anger; T score ≥60 and <70 indicates a moderate level of anger; and a T score ≥70 indicates a severe level of anger. The Cronbach’s α at the time of development was 0.92, and the Cronbach’s α of the K-PROMIS Anger scale as translated by Choi et al. [20] is 0.91.

K-IOWA

A ten-question scale developed by Loney and Milich [23], K-IOWA, is used for the screening of ADHD and the evaluation of treatment effects [24]. In this study, the K-IOWA translated by Shin et al. [15] was used, and its reliability and validity have previously been investigated. The K-IOWA consists of two subcategories: inattentive/overactive and oppositional/defiant, each containing five questions. It includes ten questions on a four-point Likert scale, with a total score ranging between 0 and 30, and higher scores indicate higher severity of inattentive-oppositional defiant behaviors. The Cronbach’s α of K-IOWA is 0.91.

K-ARS

ARS was developed by DuPaul [13] as a scale to assess children’s behaviors, and the scale was designed to assess ADHD symptoms in school-age children based on the ADHD diagnostic criteria of DSM-IV. In this study, the Korean version (K-ARS), as translated by Kim et al. [14], was used, and its reliability and validity have previously been investigated. The K-ARS comprises 18 questions in total; the total score of odd questions measures inattentiveness, and the total score of even questions measures the hyperactivity/impulsivity. Each question is rated on a three-point Likert scale according to the frequency of the respective problem behavior. The reliability and validity of the scale have been verified in previous studies. In the study by Kim et al. [14], a satisfactory internal consistency at Cronbach’s α= 0.83–0.96, with the range being dependent on age, was found.

Statistical analysis

For all statistical analyses, SPSS 20.0 (IBM Corp., Armonk, NY, USA) was used. The reliability of the Korean versions of the DSM-5 SNAP-IV inattention scale and PROMIS Anger scale was assessed by calculating the internal consistency co-efficient. To examine the concurrent validity of the SNAP-IV inattention scale, the correlation coefficients with the K-IO-WA inattention scale and K-ARS inattention scale were calculated. For the concurrent validity of the PROMIS Anger scale, the correlation coefficient with K-IOWA oppositional/defiant scale was calculated. In addition, the construct validity of the SNAP-IV inattention scale and PROMIS Anger scale was examined through factor analysis to verify whether each scale could validly assess the constructs of inattention and anger.

Reliability

The internal consistency coefficient (Cronbach’s α) of the Korean version of the SNAP-IV inattention scale was 0.94, and that of the Korean version of the PROMIS Anger scale was 0.91, indicating considerably high levels of consistency. The questions of the two scales are highly homogeneous, and the scales can thus be presumed to ensure a reliable assessment of the symptoms of anger and inattention (Table 1).

Table 1

Internal consistency coefficient of SANP-IV inattention, and PROMIS Anger scales

	Cronbach’s α
SANP-IV inattention	0.94
PROMIS Anger	0.91

SNAP-IV: the DSM-5 Level 2 Cross-Cutting Symptom Measure–in-attention for parents and guardians (Swanson, Nolan and Pelham, version IV), PROMIS: Patient-Reported Outcome Measurement Information System

Concurrent validity

Concurrent validity for the K-SNAP-IV, K-IOWA, and K-ARS inattention scales

To examine the concurrent validity of the Korean version of the SNAP-IV inattention scale, the correlations with the K-IOWA inattention and K-ARS inattention scales were an-alyzed, and the correlation coefficients were found to be 0.84 and 0.86, respectively, indicating significant correlations. The results proved that K-SNAP-IV could assess inattention in children and adolescents as reliably as the conventional K-IOWA and K-ARS inattention scales with verified reliability and validity (Table 2).

Table 2

Correlation coefficients of SNAP-IV, IOWA, and K-ARS inattention scales (n=104)

	SNAP-IV inattention	IOWA inattention	K-ARS inattention
SNAP-IV inattention	1.00
IOWA inattention	0.84*	1.00
K-ARS inattention	0.86*	0.73*	1.00

^* p <0.01. SSNAP-IV: the DSM-5 Level 2 Cross-Cutting Symptom Measure–inattention for parents and guardians (Swanson, Nolan and Pelham, version IV), IOWA: IOWA Conners Rating Scale, KARS: the Korean version of Attention-Deficit/Hyperactivity Disorder Rating Scales

Concurrent validity for the PROMIS Anger scale, K- IOWA, and K-ARS

To examine the concurrent validity of the Korean version of the PROMIS Anger scale, correlations with the K-IOWA Oppositional/Defiant scale and K-ARS hyperactive/impul-sive scale were analyzed, and the correlation coefficients were found to be 0.75 and 0.47, respectively. This indicated that the K-PROMIS could assess the oppositional and defiant behav-iors of the children and adolescents as reliably as the con-ventional K-IOWA oppositional/defiant scale. The result also implied that the anger assessed by K-PROMIS slightly differed from the hyperactive/impulsive behaviors assessed by K-ARS (Table 3). Therefore, the PROMIS Anger scale can-not validly evaluate hyperactivity/impulsivity, which are core symptoms of ADHD, but it seems to be a good test for eval-uating anger control problems in children with ADHD and for re-evaluating problems after treatment.

Table 3

Correlation of PROMIS Anger, IOWA oppositional/defiant, and K-ARS hyperactive/impulsive scales (n=104)

	PROMIS Anger	IOWA oppositional/defiant	K-ARS hyperactive/impulsive
PROMIS Anger	1.00
IOWA oppositional/defiant0.75*	1.00
K-ARS hyperactive/impulsive	0.47*	0.63*	1.00

^* p<0.01. PROMIS: Patient-Reported Outcome Measurement Information System, IOWA: IOWA Conners Rating Scale, K-ARS: the Korean version of Attention-Deficit/Hyperactivity Disorder Rating Scales

Discriminant validity

To examine the discriminant validity of the SNAP-IV in-attention and PROMIS Anger scales, the mean scores of the two scales of the five clinical groups categorized based on the DSM-V diagnostic criteria, namely 1) depressive disorder 2) anxiety, 3) ADHD, 4) ADHD+anxious depression, and 5) tic disorder+somatic symptoms, were compared (Table 4). The one-way analysis of variance (ANOVA) performed on mean scores of the SNAP-IV inattention scale for the clinical groups showed that the mean differences per group were sig-nificant (F=8.58, p<0.001). The Bonferroni test as a post-hoc test showed that the mean scores of the ADHD+anxious de-pression group were significantly higher than the mean scores of the tic disorder+somatic symptoms group, while the mean scores on the inattention scale were significantly higher for the ADHD group than for the anxiety and tic disorder+ so-matic symptoms groups (4>5; 3>2, 5). On the other hand, there was no significant difference in the mean scores of the SNAP-IV inattention scale between the ADHD and the de-pressive disorder groups, suggesting that children with de-pression show inattentive symptoms that are similar to those shown by children with ADHD. However, this may be the re-sult of the fact that the sample size of the depressive disorder group is smaller than that of the pure ADHD or ADHD with anxious depression. Therefore, there is a need to conduct a clinical study with a larger sample size to verify the result in the future. One-way ANOVA on the PROMIS Anger scale for clinical groups showed that the mean score of the ADHD+ anxious depression group tended to be higher than that of the other four groups. However, the mean differences among the groups were not significant. These results showed that the discriminant validity of the PROMIS Anger scale is low, making it difficult to discriminate children with ADHD from children with other psychiatric disorders such as depression, anxiety, and tic disorder using the PROMIS Anger scale.

Table 4

Comparison of mean score in SNAP-IV inattention and PROMIS Anger scales by groups (n=104)

	Depressiona (n=9)	Anxiety disorderb (n=14)	ADHDc (n=41)	ADHD+anxiousd depression (n=11)	Tic+somatice symptom (n=29)	F	Post hoc
SNAP-IV inattention	6.89 (5.90)	6.86 (6.11)	11.90 (5.40)	12.64 (7.27)	5.03 (4.53)	8.58*	d>e c>b, e
PROMIS Anger	4.56 (2.55)	6.64 (4.27)	6.68 (5.51)	7.18 (4.40)	3.69 (3.67)	2.45

Data are presented as mean (standard deviation). *p<0.001. SNAP-IV: the DSM-5 Level 2 Cross-Cutting Symptom Measure–inattention for parents and guardians (Swanson, Nolan and Pelham, version IV), PROMIS: Patient-Reported Outcome Measurement Information System, ADHD: attention-deficit/hyperactivity disorder

Construct validity

Factor analysis was performed on all data collected through SNAP-IV inattention and PROMIS Anger scales from 104 children and adolescents in this study. When principal com-ponent analysis was used to extract the factors, both scales were shown to be unidimensional with a single factor, ‘inat-tention’ or ‘anger.’ The construct validity of the SNAP-IV in-attention and PROMIS Anger scales was thus verified (Tables 5 and 6). Among the eight questions of the SNAP-IV inattention scale, the following were shown to most effectively assess inattention: “my child often does not follow through on instructions and fails to finish schoolwork, chore, or du-ties,” “my child often has difficulty organizing tasks and ac-tivities,” and “my child often fails to give close attention to details or makes careless mistakes in schoolwork, work, and other activities.” Among the five questions of the PROMIS Anger scale, “my child was so angry that he/she felt like yell-ing at somebody” and “my child felt mad” were shown to most effectively assess anger.

Table 5

Factor loadings of the DSM-5 Level 2 Cross-Cutting Symptom Measure–inattention for parents and guardians (Swanson, Nolan and Pelham, version IV) rating scale (n=104)

Questions	Factor loading
1. My child often fails to give close attention to details or makes careless mistakes in schoolwork, work, or other activities	0.874
2. My child often has difficulty sustaining attention in tasks or play activities	0.868
3. My child often does not seem to listen when spoken to directly	0.748
4. My child often does not follow through on instructions and fails to finish schoolwork, chores, or duties	0.880
5. My child often has difficulty organizing tasks and activities	0.877
6. My child often avoids, dislikes, or is reluctant to engage in tasks that require sustained mental effort (e.g., schoolwork or homework)	0.839
7. My child often loses things necessary for tasks or activities (e.g., toys, school assignments, pencils, books, or tools)	0.739
8. My child often is distracted by extraneous stimuli	0.831
Eigenvalues	5.56

Table 6

Factor loadings of Patient-Reported Outcome Measurement Information System Anger scale (n=104)

Questions	Factor loading
1. My child felt mad	0.884
2. My child was so angry he/she felt like yelling at somebody	0.930
3. My child was so angry he/she felt like throwing something	0.877
4. My child felt upset	0.792
5. When my child got mad, he/she stayed mad	0.833
Eigenvalues	3.73

This study investigated the reliability and validity of the Korean version of the SNAP-IV inattention scale and the PROMIS Anger scale for parents and guardians of children and adolescents aged 6–17 years. The reliability and validity of the information on the behaviors of children and adolescents provided by parents and guardians through the SNAP-IV inattention scale and the PROMIS Anger scale have been shown to be high in previous studies in other countries [23, 25, 26], while the scales were also shown to be useful in assessing the treatment effects or progress. Notably, they are time-efficient assessments of the problematic behaviors of children and adolescents, as ratings are performed in a short period. The two scales are thus expected to be useful in clinical settings where repeated measurements are required to verify the treatment effects.

The reliability coefficient of the Korean version of the SNAP-IV inattention scale was shown to be remarkably high at Cronbach’s α=0.94. The SNAP-IV inattention scale can thus be regarded as a homogeneous scale that ensures a highly reliable assessment of inattention, in particular, among the ADHD-related symptoms. Moreover, the correlation coefficients for the SNAP-IV and the inattentive/overactive score of the K-IOWA and inattention score of the K-ARS were 0.84 and 0.86, respectively, indicating high levels to support the concurrent validity of the SNAP-IV inattention scale. The children and adolescents in the ADHD+anxious depression group and the ADHD group showed significantly higher scores than the other groups on the SNAP-IV inattention scale, implying that the scale can differentiate children and adolescents with ADHD-related problems from those with other psychiatric disorders such as depression, anxiety, or tic disorder. Thus, the findings suggested that the SNAP-IV inattention scale can be not only an easy tool to use but also a reliable and valid test in treatment evaluation as well as differential screening for children and adolescents with ADHD.

The reliability coefficient of the Korean version of the PROMIS Anger scale was shown to be remarkably high at Cronbach’s α=0.91. The PROMIS Anger scale can thus be regarded as a homogeneous scale for reliably assessing anger. Moreover, the correlation coefficients for the PROMIS and the impulsive/hyperactive score of the K-ARS and the oppositional/defiant score of the K-IOWA were 0.47 and 0.75, respectively. These results suggest that the construct assessed by the PROMIS Anger scale was related to the oppositional and defiant characteristics [27] that are regarded as an attitude and behavioral response to the stimulus causing anger. The construct of the PROMIS was different from impulsive/hyperactive behaviors as the core symptoms of ADHD. In addition, the result of the analysis of the group differences in mean scores of the PROMIS Anger scale showed a trend of higher anger scores in the groups of pure ADHD or ADHD with anxiety/depression than in the depression or tic disorder+ somatic symptoms groups. Children and adolescents with ADHD repeatedly experience feelings of frustration in daily life, including in academics and social relationships [4], which may cause emotional problems such as anxiety or depression; the result may thus be taken to imply that such problems could lead to secondary behavioral problems involving oppositional and defiant behaviors. Furthermore, the mean anger score of the anxiety group was almost as high as that of the ADHD group. This may imply that anxiety in childhood and adolescence may cause emotional problems related to suppressed anger [28]. However, as the sample sizes of the anxiety and depression groups were smaller than those of the groups of pure ADHD or ADHD with anxiety/depression, the results should be verified in further studies recruiting a larger number of clinical groups. In previous studies [29,30], a high comorbidity of ADHD and tic disorder was reported, but the lowest inattention and anger scores were shown by the tic+somatic symptoms group in this study because we did not recruit patients with ADHD and tic disorder. In future studies, it is necessary to recruit children with ADHD with tic disorder to compare inattention and anger problems in children with ADHD and those in children with ADHD with tic disorder.

The factor analysis shows that the SNAP-IV inattention scale is a unidimensional scale of a set of questions for a single factor (inattention), suggesting that the scale might be useful for assessing children and adolescents with inattentive type of ADHD.

Nonetheless, a number of things should be considered when using the SNAP inattention scale or the PROMIS Anger scale. First, the two scales for parents and guardians cannot replace the direct diagnostic tests on children and adolescents, even when high scores have been obtained in these scales. A more accurate assessment and diagnosis is required based on a higher level of expertise and clinical judgment by a clinician. Furthermore, as most rating scales evaluate quantitative differences such as severity rather than the qualitative difference of symptomatic expression, the various unique characteristics of each child’s symptoms should be taken into consideration.

Finally, the limitations of this study are as follows. This study investigated the reliability and validity of the Korean version of the SNAP-IV inattention scale and the PROMIS Anger scale by comparing different clinical groups visiting the department of child and adolescent psychiatry. However, the number of participants was relatively small. In addition, the pattern of inattention and anger may vary according to the gender and age of children and adolescents. Further studies should be conducted to establish the norm scores on each scale based on age and gender for a more objective analysis of severity. Lastly, the scales examined in this study are rating scales for parents and guardians; therefore, there is a possibility that the subjective discomfort and severity of experience in children and adolescents with ADHD could not be sufficiently reflected. Thus, the DSM-5 Level 2 self-reporting scale for children and adolescents and the rating scale for parents and guardians should be applied together to further increase their utility as tools for screening and treatment.

The reliability and validity of the Korean version of DSM-5 Level 2 Cross-cutting Symptom Measure inattention and anger for parents and guardians of children aged 6–17 years were examined by targeting the parents and guardians of the clinical group of children and adolescents visiting the department of child and adolescent psychiatry at a university hospital located in Seoul, Korea. The results showed high levels of reliability and validity for the two scales, which are expected to be useful, in particular, as tools for not only an efficient screening of ADHD in clinical practice but also for better differentiation of children and adolescents with ADHD from those with emotional problems such as depression, anxiety, or tic disorder. The findings suggested that the DSM-5 inattention and anger scales for parents and guardians would be useful in clinical settings, mental health care service, and re-search as easy, economic assessment tools.

This study was conducted by the fund of the Korean Ministry of Health and Welfare (800-20200053).

The authors have no potential conflicts of interest to disclose.

Conceptualization: Min-Sup Shin. Data curation: all authors. Formal analysis: all authors. Funding acquisition: Min-Sup Shin, Bung-Nyun Kim. Investigation: all authors. Methodology: Min-Sup Shin, Minji Cho, Mirae Jang, Hanbyul Shin. Project administration: Min-Sup Shin, Minji Cho, Mirae Jang, Hanbyul Shin. Resources: all authors. Supervision: Min-Sup Shin. Writing—original draft: Min-Sup Shin, Minji Cho, Mirae Jang, Hanbyul Shin. Writing—review & editing: Min-Sup Shin, Minji Cho, Mirae Jang, Hanbyul Shin.