Quality Analysis of Intelligence Structure Test 2000 Revision (IST 2000R) Items in Indonesian


Medianta Tarigan(1*), Fadillah Fadillah(2)

(1) Department of Psychology, Universitas Pendidikan Indonesia, Bandung, Indonesia
(2) Department of Visual Communication Design, Institut Teknologi Bandung, Indonesia
(*) Corresponding Author

Abstract


This study aimed to test the quality of item analysis in the Indonesian IST 2000R using the Item Response Theory (IRT) method and participants comprising 1780 students aged 16-20. IRT 3PL analysis and DIF examination showed that the Indonesian IST 2000R had good item quality, with an improvement in IST 70. The discriminatory power parameter (ɑ) of the verbal category items was high. Furthermore, the difficulty level parameter (b) indicated that more than 50% of the items on the IST 2000R are in the average category. In the guessing probability parameter (c), 8 of 9 subtests showed that more than 50% of the items are in a low category. Overall, the Indonesian IST 2000R has a small gender bias because all subtests obtained more than 60% in category A.

Keywords


Intelligence Structure Test (IST), Item Response Theory, item analysis

Full Text:

PDF

References


Abedalaziz, N., Leng, C. H., & Alahmadi, A. (2014). Detecting a gender-related differential item functioning using transformed item difficulty. Malaysian Online Journal of Educational Sciences, 2, 16-22.

Adedoyin, O. O., & Mokobi, T. (2013). Using IRT psychometric analysis to examine the quality of junior certificate mathematics multiple choice test items. International Journal of Asian Social Science, 3(4), 992–1011.

Adinugroho, I. (2016). Pengujian properti psikometrik intelligenz struktur test subtes kemampuan spasial dua dimensi (form auswahl): Studi pada dua SMA swasta di Jakarta. Jurnal Ilmiah Psikologi MANASA, 5(2).

Ayanwale, M. A. (2019). Efficacy of item response theory in the validation and score ranking of dichotomous and polytomous response mathematics achievement tests in Osun State, Nigeria (Doctoral Thesis Unpublished). Institute of Education, University of Ibadan.

Baker, F. B. (2001). The basics of item response theory second edition. ERIC Clearinghouse on Assessment and Evaluation.

Beauducel, A., Brocke, B., & Liepmann, D. (2001). Perspectives on fluid and crystallized intelligence: Facets for verbal, numerical, and figural intelligence. Personality and Individual Differences, 30(6), 977–994. https://doi.org/10.1016/S0191-8869(00)00087-8

Beauducel, A., Liepmann, D., Horn, S., & Brocke, B. (2010). Intelligence structure test. Hogrefe.

Cain, M. K., Zhang, Z., & Yuan, K. H. (2017). Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence, and estimation. Behavior Research Methods, 49(5), 1716–1735. https://doi.org/10.3758/s13428-016-0814-1

Cohen, & Swerdlik. (2010). Psychological testing and assessment: An introduction to tests and. measurement, seventh edition. McGraw-Hill.

Foster, G. C., Min, H., & Zickar, M. J. (2017). Review of item response theory practices in organizational research: Lessons learned and paths forward. Organizational Research Methods, 20(3), 465-486. https://doi.org/10.1177/1094428116689708

Gierl, M. J., Bisanz, J., Bisanz, G. L., Boughton, K. A., & Khaliq, S. N. (2001). Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20(2), 26-36. https://doi.org/10.1111/j.1745-3992.2001.tb00060.x

Ibrahim, A. (2018). Differential item functioning: The state of the art. Jigawa Journal of Multidisciplinary Studies (JJMS), 1(1), 37-50.

Martín, E. S., Del Pino, G., & De Boeck, P. (2006). IRT models for ability-based guessing. Applied Psychological Measurement, 30(3), 183–203. https://doi.org/10.1177/0146621605282773

McGrory, S., Doherty, J. M., Austin, E. J., Starr, J. M., & Shenkin, S. D. (2014). Item response theory analysis of cognitive tests in people with dementia: A systematic review. BMC Psychiatry. https://doi.org/10.1186/1471-244X-14-47

Meyer, J. P. (2014). Applied measurement with jmetrik. Routledge. https://doi.org/10.4324/9780203115190

Obinne, A. D. E. (2012). Using IRT in determining test items prone to guessing. World Journal of Education, 2(1). https://doi.org/10.5430/wje.v2n1p91

Ogunsakin, I. B., & Shogbesan, Y. O. (2018). Item response theory (IRT): A modern statistical theory for solving measurement problems in the 21st century. International Journal of Scientific Research in Education (IJSRE), 11(3B), 627-635.

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1–11. https://doi.org/10.1080/2331186X.2017.1301013

Rahmawati, E. (2014). Evaluasi karakteristik psikometri intelligenz struktur test (IST) [Proceeding]. Seminar Nasional Psikometri, UMS, Indonesia.

Reilly, D., & Neumann, D. L. (2013). Gender-role differences in spatial ability: A meta-analytic review. Sex Roles, 68(9–10), 521–535. https://doi.org/10.1007/s11199-013-0269-0

Reilly, D., Neumann, D. L., & Andrews, G. (2022). Gender differences in self-estimated intelligence: Exploring the male hubris, female humility problem. Frontiers in Psychology, 13, 812483. https://doi.org/10.3389/fpsyg.2022.812483

Ring, P., Neyse, L., David-Barett, T., & Schmidt, U. (2016). Gender differences in performance predictions: Evidence from the cognitive reflection test. Frontiers in Psychology, 7, 1680. https://doi.org/10.3389/fpsyg.2016.01680

Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction, 11(3), 557–572. https://doi.org/10.12973/iji.2018.11338a

Schulze, D., Beauducel, A., & Brocke, B. (2005). Semantically meaningful and abstract figural reasoning in the context of fluid and crystallized intelligence. Intelligence, 33(2), 143–159. https://doi.org/10.1016/j.intell.2004.07.011

Sirodj, D. A. N. (2018). Analisis kualitas item intelligence structure test (IST) melalui metode item response theory (IRT). SCHEMA (Journal of Psychological Research), 4(2), 98–108. https://doi.org/10.29313/schema.v4i2.4420

Tarigan, M., & Fadillah. (2021a). Properti pikometrik intelligenz struktur test subtes kemampuan numerik (rechenaufgaben dan zahlen reihen). Jurnal Psikologi Ilmiah, 13(2), 155–170. https://doi.org/10.15294/intuisi.v13i2.31839

Tarigan, M., & Fadillah, F. (2021b). Properti psikometri struktur inteligensi IST subtes verbal (Satzergaenzung, Wortauswahl, dan Analogien) berbahasa Indonesia. Jurnal Muara Ilmu Sosial, Humaniora, dan Seni, 5(1), 63-72. https://doi.org/10.24912/jmishumsen.v5i1.9623.2021

Yacob, A., Ali, N., Yusoff, M. H., MohdSaman, M. Y., & Hamzah, W. M. (2014). Personalized learning: An analysis using item response theory. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 8, 1107-1113.




DOI: https://doi.org/10.15575/psy.v9i1.14977

Refbacks

  • There are currently no refbacks.


web stats

View My Stats

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.