Development and Content Validity of Test for the HOTRTG (Higher Order Thinking Research Test for Graduates) in the Subject of Research Methods at the University level in Punjab, Pakistan

Abstract

Tests for Higher order thinking (HOT) skills are available around the globe. In Pakistan, the quality of tests used for assessing HOT is very rare. Specialized tests are required to measure the HOT of students for which this study was made. It aimed to 1) develop a multiple choice and open-ended question test for measuring the HOT of graduate students, and 2) ensure the content validity and reliability of the test. This was instrumental research consisting of three phases; 1) instrument development, 2) expert validation, and 3) pilot testing. The instrument was developed in Multiple Choice Questions (MCQs) and Open-Ended Questions (OEQs) format. The developed instrument consisted of 38 items (30 MCQs, 8 OEQs). The CVR value for the test ranged from 0.571 to 1.000 and the CVI value for the test was 0.774 to ensure the validity of the test. The tool was validated by expert opinion on the material, construction, and language aspects which showed that all 38 items (30 MCQs and 8 OEQs) were valid and of very good quality. At the pilot testing phase, it was conducted on 80 graduate students of the faculty of Humanities and Social Sciences, KFUEIT, R Y Khan, Punjab, Pakistan. The tested instrument was then analyzed to determine the level of difficulty, discrimination index, and reliability of the test. On the bases of the results of difficulty level and discrimination index 26 questions (20 MCQs and 6 Short Questions) were valid and 10 MCQ items and 2 OEQ items were found below the criteria which were dropped from the final draft of the test. The Alpha value of reliability of the test was 0.735 which indicates that the test is highly reliable. On the bases of the results, the developed instrument proved valid to measure the HOT skills of graduate students in the subject of research at the University level in Punjab, Pakistan.