Beginning with the projectively invariant method for linear programming, interior point methods have led to powerful algorithms for many difficult computing problems, in combinatorial optimization, logic, number theory and non-convex optimization. Algorithms for convex optimization benefitted from many pre-established ideas from classical mathematics, but non-convex problems require new concepts. Lecture series I am presenting at the conference on Foundations of Computational Mathematics, 2014, outlines some of these concepts{computational models based on the concept of the continuum, algorithms invariant w.r.t. projective, bi-rational, and bi-holomorphic transformations on co-ordinate representation, extended proof systems for more efficient certificates of optimality, extensions of Grassmanns extension theory, efficient evaluation methods for the effect of exponential number of constraints, theory of connected sets based on graded connectivity, theory of curved spaces adapted to the problem data, and concept of relatively algebraic sets in curved space. Since this conference does not have a proceedings, the purpose of this article is to provide the material being presented at the conference in more widely accessible form.