#### Classes of ODE solutions: smoothness, covering numbers, implications for noisy function fitting, and the curse of smoothness phenomenon

##### Ying Zhu, Mozhgan Mirzaei

Numerical methods for recovering ODE solutions from data largely rely on approximating the solutions using basis functions or kernel functions under a least square criterion. The accuracy of this approach hinges on the smoothness of the solutions. This paper provides a theoretical foundation for these methods by establishing novel results on the smoothness and covering numbers of ODE solution classes (as a measure of their "size"). Our results provide answers to "how do the degree of smoothness and the "size" of a class of ODEs affect the "size" of the associated class of solutions?" We show that: (1) for the first order ODEs, if the absolute values of all $k$th order derivatives are bounded by $1$, then the solution can end up with derivatives whose magnitude grows factorially fast - "a curse of smoothness"; (2) our upper bounds for the covering numbers of the $(\beta+2)-$degree smooth solution classes are greater than those of the "standard" $(\beta+2)-$degree smooth class of univariate functions; (3) the mean squared error of least squares fitting in noisy settings has a convergence rate no larger than $\left(\frac{1}{n}\right)^{\frac{2\left(\beta+2\right)}{2\left(\beta+2\right)+1}}$ if $n=\Omega\left(\left(\beta\sqrt{\log\left(\beta\vee1\right)}\right)^{4\beta+10}\right)$, and under this condition, the rate $\left(\frac{1}{n}\right)^{\frac{2\left(\beta+2\right)}{2\left(\beta+2\right)+1}}$ is minimax optimal in the case of $y^{'}\left(x\right)=f\left(x,\,y\left(x\right)\right)$; (4) more generally, for the higher order Picard type ODEs, $y^{\left(m\right)}\left(x\right)=f\left(x,\,y\left(x\right),\,y^{'}\left(x\right),\,...,y^{\left(m-1\right)}\left(x\right)\right)$, the covering number of the solution class is bounded above by the product of the covering number of the class $\mathcal{F}$ that $f$ ranges over and the covering number of the set where initial values lie.

arrow_drop_up