We present a novel (high-order) hybridizable discontinuous Galerkin (HDG) scheme for the fluid-structure interaction (FSI) problem. The (moving domain) incompressible Navier-Stokes equations are discretized using a divergence-free HDG scheme within the arbitrary Lagrangian-Euler (ALE) framework. The nonlinear elasticity equations are discretized using a novel HDG scheme with an H(curl)-conforming velocity/displacement approximation. We further use a combination of the Nitsche's method (for the tangential component) and the mortar method (for the normal component) to enforce the interface conditions on the fluid/structure interface. A second-order backward difference formula (BDF2) is use for the temporal discretization. Numerical results on the classical benchmark problem by Turek and Hron show a good performance of our proposed method.