The purpose of this research was to adapt and clarify the criteria of the Greek general assessment framework of the Presidential Decree (P.D.)152/2013 in the physical education subject and to examine the tool’s validity and reliability. Secondary aims were a) the discrimination of the teachers according to their observed performance, and b) the examination of the effect of school grade in teaching effectiveness. The P.D. 152/2013 includes five categories analyzed in 15 criteria on a mixed four-level rubric (Incomplete, Sufficient, Very Good, Excellent). In the current research, the 10 observable criteria of three categories that relate to teaching were used. The criteria were analyzed in detail, clarified and adapted, according to international literature on physical education (PE) teaching effectiveness. Following the step-by-step upgrade of the rubric’s levels (the same indicator at all levels but each time with more qualitative characteristics), 41 indicators were developed. In order to examine the validity and reliability of the adapted tool, trained evaluators observed in person 40 PE lessons taught in early and in upper grades by 20 PE teachers. The lessons were assessed in every indicator of the rubric, and classified in one of the four performance levels. Principal Component Analysis, descriptive statistics, chi-squared and ANOVA tests were carried out to examine the construct validity of the tool. The results revealed that the adapted rubric to the PE subject included the three initial categories, seven criteria and 26 indicators and can be considered as a valid and reliable instrument for assessing PE teachers’ effectiveness.