The output impedance matrix of a grid-connected converter plays an important role in analyzing system stability. Due to the dynamics of the DC-link control and the phase locked loop (PLL), the output impedance matrices of the converter and grid are difficult to be diagonally decoupled simultaneously, neither in the dq domain nor in the phase domain. It weakens the effectiveness of impedance-based stability criterion (ISC) in system oscillation analysis. To this end, this paper innovatively proposes the generalized-impedance based stability criterion (GISC) to reduce the dimension of the transfer function matrix and simplify system small-signal stability analysis. Firstly, the impedances of the converter and the grid in polar coordinates are formulated, and the concept of generalized-impedance of the converter and the grid is put forward. Secondly, through strict mathematical derivation, the equation that implies the dynamic interaction between the converter and the grid is then extracted from the characteristic equation of the grid-connected converter system. Using the proposed method, the small-signal instability of system can be interpreted as the resonance of the generalized-impedances of the converter and the grid. Besides, the GISC is equivalent to ISC when the dynamics of the outer-loop control and PLL are not considered. Finally, the effectiveness of the proposed method is further verified using the MATLAB based digital simulation and RT-LAB based hardware-in-the-loop (HIL) simulation.