Security and Privacy concerns in Radio frequency identification (RFID) technology particularly RFID Card, is a wide research area which have attracted researchers for over a decade. Authenticating users at the Card end of the RFID technology constitutes one of the major sources of attacks on the system. In this research, we studied the various known attacks and mitigation available. We proposed a conceptual framework that that can be used to mitigate the unauthorized use of RFID Card. This concept will mitigate the single point of the RFID card failure: unauthorized use.