Handover optimization in O-RAN faces growing challenges due to heterogeneous user mobility patterns and rapidly varying radio conditions. Existing ML-based handover schemes typically operate at the near-RT layer, which lack awareness of the mobility-mode and struggle to incorporate a longer-term predictive context. This paper proposes a multi-modal mobility-aware optimization framework in which all predictive intelligence, including mobility mode classification, short-horizon trajectory and RSRP forecasting, and a PPO Actor--Critic policy, runs entirely inside an rApp in the non-RT RIC. The rApp generates per-UE ranked neighbour-cell recommendations and delivers them to the existing handover xApp through the A1 interface. The xApp combines these rankings with instantaneous E2 measurements and performs the final standards-compliant handover decision. This hierarchical design preserves low-latency execution in the xApp while enabling the rApp to supply richer and mode-specific predictive guidance. Evaluation using mobility traces demonstrates that the proposed approach reduces ping-pong handover events and improves handover reliability compared to conventional 3GPP A3-based and ML-based baselines.