The World Bank’s Human Capital Index (HCI) aims to provide new information regarding future productivity of each country’s workforce, by synchronizing available International Large-Scale Assessment (ILSA) and regional test program results. Linking the literature on ILSA participation, this study questions the problematic nature of this approach and revisits the comparability issue of ILSA results. We find that education systems are imposed upon a score penalty depending on which ILSA or regional test program they choose to partake in. In particular, our results show that (i) test-overlap systems used in the score synchronization procedure are systematically different from systems that only choose to participate in one ILSA or exclusively in regional tests, (ii) inter-test score exchange rate is volatile due to sampling design and cohort effects, (iii) test participation type alone accounts for about 57.8 percent of the variation in synchronized scores, and the score penalty is especially salient for systems that exclusively participate in regional test programs; the majority of which are low-income and lower-middle income countries. Findings in this study show how various intra- and extrapolations to compensate for missing data in effect introduce large score penalties for systems that either did not participate or only partially participated in ILSAs. Finally, this study contributes to research on reasons for participation in ILSAs and the global rise of test-based accountability reform, under which the World Bank’s new HCI may be seen as a tool to incentivize participation in ILSAs by penalizing those governments that have chosen alternative, non-standardized paths for measuring learning outcomes of students.