Promoting or Suppressing? The Impact of Generative AI on Students' Programming Ability from a Meta-Analysis Based on 42 Experiments and Quasi-Experiments

Generative AI has been widely used in programming education due to its real-time personalized feedback, code parsing, and debugging capabilities. However, there is no consensus in academia on whether generative AI can enhance students' programming abilities. This article conducts a meta-analysis of 42 experimental and quasi experimental studies after ChatGPT3.5 and finds that: 1) Generative AI has a moderate promoting effect on students' overall programming ability, with the greatest promoting effect on operational ability, followed by cognitive ability; 2) Generative AI has a promoting effect on students' programming ability in different experimental cycles, and the use of generative AI has the most significant impact on students' programming ability in the short term; 3) Compared to the basic education stage, generative AI has a more significant promoting effect on students' programming ability in higher education stage; 4) In medium-sized studies, generative AI has the best impact on students' operational ability, while in small-scale studies, the impact of generative AI on students' overall programming ability is more prominent; 5) Students without programming experience who use generative AI are more likely to enhance their programming abilities, while those with experience are more susceptible to metacognitive risks. Based on this research conclusion, this article proposes the following suggestions for programming teaching practice: 1) Strengthen the cultivation of students' operational abilities and guide their participation in deep cognition; 2) effectively leverage the short-term effects of generative AI on programming ability and strategically plan long-term pathways for integration; 3) Continuously deepen AI empowerment in higher education, and focus on guiding students to apply it reasonably in basic education; 4) select an appropriate teaching scale based on instructional objectives; 5) establish AI scaffolding for beginners and set metacognitive activation points for experienced learners.