The goal of this paper is to give a comprehensive and short review on how to compute the first and second order topological derivative and potentially higher order topological derivatives for PDE constrained shape functionals. We employ the adjoint and averaged adjoint variable within the Lagrangian framework and compare three different adjoint based methods to compute higher order topological derivatives. To illustrate the methodology proposed in this paper, we then apply the methods to a linear elasticity model. We compute the first and second order topological derivative of the linear elasticity model for various shape functionals in dimension $2$ and $3$ using Amstutz' method, the averaged adjoint method and Delfour's method. In contrast to other contributions regarding this subject, we not only compute the first and second order topological derivative, but additionally give some insight on various methods and compared their applicability and efficiency with respect to the underlying problem formulation.