Emotions have a major interactive role in defining how humans interact with their environment by encoding their perception to external events and accordingly, influencing their cognition and decision-making process. Therefore, increasing attention has been directed toward integrating human affective states into system design in order to optimize the quality of task performance. In this work, we seize on the significant correlation between emotions and behavioral performance that is reported in several psychological studies and develop an online closed-loop design framework for Human-Robot Interaction (HRI). The proposed approach monitors the behavioral performance based on the levels of Pleasure, Arousal, and Dominance (PAD) states for the human operator and when required, applies an external stimulus which is selected to induce an improvement in performance. The framework is implemented on an HRI task involving a human operator teleoperating an articulated robotic manipulator. Our statistical analysis shows a significant decrease in pleasure, arousal, and dominance states as the behavioral performance deteriorates $(p < 0.05)$. Our closed-loop experiment that uses an audio stimulus to improve emotional state shows a significant improvement in the behavioral performance of certain subjects.