With the large-scale application of industrial robots, the safety problems caused by the sudden failures of industrial robots occur from time to time. Traditional fault diagnosis methods based on data analysis have several problems, such as the sensor data are disturbed by the external environment, the communication protocols for different types of robots are complex and not unified, the monitoring systems embedded in the execution systems affect each other, and so on. In this paper, we propose a vision-based fault detection method for industrial robots. According to this method, we perform real-time analysis of operation video from industrial robots. An image segmentation module extracts the body part of an industrial robot, and an image hashing module then generates the posture code describing the movement of the industrial robot. Thus, we can obtain early warnings based on abnormal movements of the robot, which can be detected by the sequence pattern analysis technology. The proposed method does not rely on communication protocols of industrial robots and can monitor industrial robots in a real-time non-contact manner. Besides, the method is easy for deployment at a low cost. We conduct experiments on a collected dataset containing videos of industrial robots. Experiment results show that the proposed method can accurately identify the abnormal actions of industrial robots at an average detection accuracy of 99.15%, which meets the needs of industrial applications in practice.