Safety is a key issue in human-robot interaction, but perceived safety has not been well-studied in comparison to physical safety. In this paper, we address the multidisciplinary perspective of perceived safety in human-robot interaction. To investigate how the comfort of the user, sense of control of the user, unpredictable robot behaviors, and trust impact the safety perception of the user, we designed a randomized controlled within-subject experiment. We devised five different experimental conditions in which we investigated the relationships between perceived safety and comfort, sense of control, and trust. In each condition, we modified one factor. To extend our previous findings, the participants were asked to answer questionnaires that measure comfort, sense of control, trust, and perceived safety. The questionnaire results show a strong correlation between these factors and the perceived safety. Since these factors are the main factors that influence perceived safety, they should be considered in human-robot interaction design decisions. The effect of individual characteristics such as personality and gender on perceived safety was also discussed. Moreover, we analyzed the facial affect and physiological signals of the participants for predicting perceived safety from objective measures. The data from objective measures revealed that physiological signals give better prediction of perceived safety rather than facial affect data. We believe this article can play an important role in the goal of better understanding perceived safety in human-robot interaction.