Internal Consistency and Item-Level Validity of a Six-Construct Leadership-Attributes Scale
Main Article Content
Abstract
Psychometric instruments used in comparative educational research are frequently deployed in a main study on the strength of pilot-study reliability alone, with little scrutiny of whether that reliability replicates at scale. This paper reports such a case. A pilot study of a 30-item, six-construct leadership-attributes questionnaire (self-efficacy, resilience, emotional intelligence, cognitive competence, social preference, perceived leadership behaviour) reported Cronbach's alpha of 0.60–0.91 across the six constructs. When the same instrument was administered to a substantially larger main sample (N = 450; 259 Military School and 191 Non-Military School students), internal consistency collapsed: alpha ranged from −0.09 to 0.05 across all six constructs, and inter-item/inter-construct correlations were uniformly weak. This reliability failure is reported in full, some potential explanations are explored (contextual item interpretation, variability in adolescent responses, and construct multi-dimensionality), and it is shown that the findings of an analysis of the items (not the composite scores) are interpretable and theoretically coherent even if the composite score is not. We assert that item-level reporting should be used as a must, and not an optional add-on, to the composite reliability when composite reliability is near the margin or unknown at the time of analysis of the main study.