In this work, we consider the adaptive nonlinear control problem for strict feedback nonlinear systems, where the functions that determine the dynamics of the system are completely unknown. We assume that certain upper bounds for the functions $g_i$s of the system are known. The objective of the control design is to design an adaptive controller that can adapt to changes in the unknown functions that are even abrupt. We propose a novel backstepping memory augmented NN (MANN) adaptive control method for the control of strict feedback non-linear systems. Here, each NN, in the backstepping NN adaptive controller, is augmented with an external working memory. The NN can write relevant information to its working memory and later retrieve them to modify its output, thus providing it with the capability to leverage past learned information effectively and improve its speed of learning. We propose a specific design for this external memory interface and show that the proposed control design achieves bounded stability for the closed loop system. We also provide substantial numerical evidence showing that the proposed memory augmentation improves the speed of learning by a significant margin.