Objectives: Acute kidney injury (AKI) is a critical clinical syndrome requiring immediate intervention. Our previous research developed an artificial intelligence (AI) prediction model for AKI. This study aims to evaluate the impact of this AI model on improving the predictive capabilities of AKI prediction. Methods: We utilized convolutional neural networks with a residual block to predict AKI in hospitalized patients. The training set comprised data from 183,221 patients at Seoul National University Hospital (2013-2017). At Seoul National University Bundang Hospital (2020-2021), we randomly selected 74 patients from departments with high AKI rates, including 15% AKI cases. We assessed the impact of an AI model on clinical decisions by comparing evaluations with and without AI assistance. Results: Accuracy was highest for physicians, with students and the AI model were similar (physician: 0.797, student: 0.574, AI: 0.568). AI assistance improved recall and F1 scores for all almost individuals (recall: 52.4% to 71.4%, F1: 37.7% to 46.1%). In the AKI predicted group, recall increased while F1 decreased for physicians (recall: 36.4% to 60%, F1: 43.2% to 33.3%) and students (recall: 54.5% to 80%, F1: 44.4% to 36.9%). For the non-AKI predicted group, both saw significant gains in recall and F1 with AI (physicians: recall 16.7% to 87.5%, F1: 18.2% to 66.7%; students: recall 44.4% to 75%, F1: 21.1% to 40%). Review times decreased for all with AI (median: 69.0 to 52.0 seconds, p=0.032), especially in the non-AKI predicted group (68.5 to 46.0 seconds, p<0.001; AKI predicted group 71.0 to 57 seconds, p<0.001). Conclusions: AI notably improved physician performance, especially in the non-AKI predicted group with significant time savings and higher F1 scores. However, its impact was less marked in the AKI-predicted group. Alongside enhancing AI, studies on its application and target groups are essential