The research of removing rain from pictures or videos has always been an important topic in the field of computer vision and image processing. Most noise reduction methods more or less remove texture details in rain-free areas, resulting in an over-smoothing effect in the restored background. The research on image noise removal is very meaningful. We exploit the powerful generative power of a modified generative adversarial network (CGAN) by enforcing an additional condition that makes the derained image indistinguishable from its corresponding ground-truth clean image. An efficient and lightweight attention machine mechanism NAM is introduced in the generator, and an IDN-CGAN model is proposed to capture image salient features through attention operations. Taking advantage of the mutual information in different dimensions of the features to further suppress insignificant channels or pixels to ensure better visual quality, we also introduce a new fine-grained loss function in the generator-discriminator pair, predicting and real data degree of disparity to achieve improved results.