This work presents the current collection of mathematical models related to neural networks and proposes a new family of such with extended structure and dynamics in order to attain a selection of cognitive capabilities. It starts by providing a basic background to the morphology and physiology of the biological and the foundations and advances of the artificial neural networks. The first part then continues with a survey of all current mathematical models and some of their derived properties. In the second part, a new family of models is formulated, compared with the rest, and developed analytically and numerically. Finally, important additional aspects and any limitations to deal with in the future are discussed.