This article introduces HODLR2D, a new hierarchical low-rank representation for a class of dense matrices arising out of $N$ body problems in two dimensions. Using this new hierarchical framework, we propose a new fast matrix-vector product that scales almost linearly. We apply this fast matrix-vector product to accelerate the iterative solution of large dense linear systems arising out of radial basis function interpolation and discretized integral equation. The space and computational complexity of HODLR2D matrix-vector products scales as $\mathcal{O}(pN \log(N))$, where $p$ is the maximum rank of the compressed matrix subblocks. We also prove that $p \in \mathcal{O}(\log(N)\log(\log(N)))$, which ensures that the storage and computational complexity of HODLR2D matrix-vector products remain tractable for large $N$. Additionally, we also present the parallel scalability of HODLR2D as part of this article.