Delays are an important phenomenon arising in a wide variety of real world systems. They occur in biological models because of diffusion effects or as simplifying modeling elements. We propose here to consider delayed stochastic reaction networks. The difficulty here lies in the fact that the state-space of a delayed reaction network is infinite-dimensional, which makes their analysis more involved. We demonstrate here that a particular class of stochastic time-varying delays, namely those that follow a phase-type distribution, can be exactly implemented in terms of a chemical reaction network. Hence, any delay-free network can be augmented to incorporate those delays through the addition of delay-species and delay-reactions. Hence, for this class of stochastic delays, which can be used to approximate any delay distribution arbitrarily accurately, the state-space remains finite-dimensional and, therefore, standard tools developed for standard reaction network still apply. In particular, we demonstrate that for unimolecular mass-action reaction networks that the delayed stochastic reaction network is ergodic if and only if the non-delayed network is ergodic as well. Bimolecular reactions are more difficult to consider but an analogous result is also obtained. These results tell us that delays that are phase-type distributed, regardless of their distribution, are not harmful to the ergodicity property of reaction networks. We also prove that the presence of those delays adds convolution terms in the moment equation but does not change the value of the stationary means compared to the delay-free case. Finally, the control of a certain class of delayed stochastic reaction network using a delayed antithetic integral controller is considered. It is proven that this controller achieves its goal provided that the delay-free network satisfy the conditions of ergodicity and output-controllability.