In recent times, the research works relating to smart traffic infrastructure have gained serious attention. As a result, research has been carried out in multiple directions to ensure that such infrastructure can improve upon our existing (mostly) human-controlled traffic infrastructure, without violating the safety margins. For this reason, cyber security issues of such infrastructure are of paramount interest. Keeping this in mind, we conduct a review of existing models, their vulnerabilities and how such vulnerabilities can be handled. Our work covers a vast area from the domain of security, starting from the theoretical notions of cryptography to the real-life adaptation of them. At the same time, we also consider the security issues that may arise due to the usage of artificial intelligence/machine learning in the infrastructure. We believe that our work will help future researchers to gain a comprehensive yet concise look at cyber security for smart traffic infrastructure.