In this paper, the fault diagnosis (FD) problem in image-based visual servoing with eye-in-hand configurations is investigated. The potential failures are detected and isolated based on approximating parameters related. First, the failure scenarios of the visual servoing systems are reviewed and classified into the actuator and sensor faults. Second, a residual generator is proposed to detect the failure occurrences, based on the Kalman filter. Third, a decision table is proposed to isolate the fault type. Finally, simulation and experimental results are given to validate the efficacy and the efficiency of the proposed FD strategies.