Students’ achievement in core subjects in the West African Secondary School Certificate Examination (WASSCE) has been a subject of concern for stakeholders due to poor performance. The purpose of this study is to assess whether item difficulty is a significant source of variability to the measurement of students’ achievement in WASSCE in Ghana, using the generalizability theory. The study had three specific objectives: (1) to examine the sources of variability (e.g., item) to students’ achievement in WASSCE in Ghana, (2) to assess the dependability of students’ responses in relation to their ability, (3) evaluate how many items are sufficient to provide an optimum measure of student achievement. The one-facet crossed random design was adopted as the research design. Data were obtained from students’ achievement in 2015 WASSCE in the four subjects, namely, Mathematics, Science, Social Studies and English Language. The generalized analysis of variance (GENOVA) was used for the analysis, conducting both G-study and D-study. The analysis revealed that the findings from this study showed that item difficulty had minimal effect on the variability in students’ achievement in English Language and Mathematics. The result further revealed that item difficulty had a significant effect on variability in students’ achievement in Science and Social Studies. Only the English Language test showed relatively low reliability. The study recommended the item difficulty and content structure of the Science and Social Studies multiple-choice test should be evaluated by WAEC officials and their examiners.