Reinforcement learning (RL) and brain-computer interfaces (BCI) are two fields that have been growing over the past decade. Until recently, these fields have operated independently of one another. With the rising interest in human-in-the-loop (HITL) applications, RL algorithms have been adapted to account for human guidance giving rise to the sub-field of interactive reinforcement learning (IRL). Adjacently, BCI applications have been long interested in extracting intrinsic feedback from neural activity during human-computer interactions. These two ideas have set RL and BCI on a collision course for one another through the integration of BCI into the IRL framework where intrinsic feedback can be utilized to help train an agent. This intersection has created a new and emerging paradigm denoted as intrinsic IRL. To further help facilitate deeper ingratiation of BCI and IRL, we provide a tutorial and review of intrinsic IRL so far with an emphasis on its parent field of feedback-driven IRL along with discussions concerning validity, challenges, and open problems.