Different tougheners including methyl methacrylate-butadiene-styrene terpolymer (MBS, core-shell type), maleic anhydride (MAH) grafted ethylene-octene copolymer (EOM), and MAH grafted polyethylene wax (PEM) were investigated for toughening the polycarbonate (PC) composites reinforced by short carbon fiber (SCF) and flake graphene (FG). The effects of tougheners on the preparation, thermal conductivity and mechanical properties of PC composites were studied. Scanning electron microscopy was used for characterizing the impact fracture surfaces of the composites. The results showed that introducing tougheners into the carbon reinforced PC composites was beneficial to improving the processability, and PEM was more effective than EOM and MBS. Meanwhile, the through-thickness and the in-plan thermal conductivity decreased to some degree due to the isolated island effects of tougheners. Moreover, the brittle PC composites with high flexural stress could be easily toughened by tougheners. In contrast, PEM had better toughening function than EOM and MBS, and correspondingly, the stiffness of the composites was the lowest for the PEM toughened systems. The fractography revealed that dense and uniformly distributed carbon fillers dispersed in matrix PC and circular cavities coexisted in the composites. The naked fiber length gradually increased as the ductility of composite materials improved.