Deep Reinforcement Learning (DRL)-based power system optimal dispatch, which is often modeled as Transient Security-Constrained Optimal Power Flow (TSC-OPF), trains efficient dispatching agents that can adapt to different scenarios and provide control strategies quickly. However, three typical issues seriously affect the training efficiency and the performance of the dispatch agent, namely, the difficulty of quantifying the transient instability level, the high dimensionality of the state space and action space, and the frequent generation of actions that correspond to non-convergent power flows during the early training stage. To address these issues, a fast-converged DRL method for TSC-OPF is proposed in this paper. Firstly, a transient security constraint transcription method based on the simulation time duration of instability is proposed to quantify the instability level. Secondly, a general method for Markov decision process modeling of TSC-OPF is proposed to decrease the dimensionality of the observation space. Finally, two general improvement techniques for off-policy DRL algorithms are proposed. A warm-up training technique is introduced to improve the efficiency of agents learning how to generate actions that lead to convergent power flows. A parallel exploration technique is adopted to improve the efficiency of agents exploring the action space. Based on the above studies, environments for TSC-OPF with the objectives of minimizing generation cost and minimizing control cost are constructed and dispatch agents are built and trained. The proposed method is tested in the IEEE 39-bus system and a practical 710-bus regional power grid. Test results show that the training process converges rapidly, the success rate of dispatch in both cases exceeds 99.70 percent, and the decision-making costs very little time, which verifies the effectiveness and efficiency of the proposed method.