A Triangular Network For Density Estimation

Xi-Lin Li

In this paper, triangular networks refer to feedforward neural networks with triangular block matrices as their connection weights, and they are studied for density estimation. A special two layer triangular monotonic neural network unit is designed and shown to be universal approximator for invertible mappings with triangular Jacobians based on the simple observation that positively weighted sum of monotonically increasing functions is still monotonic. Then, deep invertible neural networks consisting of stacked such monotonic triangular network units and permutations are proposed as universal density estimators. Our method is most closely related to neural autoregressive density estimations, especially the block neural autoregressive flow. But, unlike many autoregressive models, our designs are highly modular, parameter economy, computationally efficient, and applicable to density estimation of data with high dimensions. Experimental results on image density estimation benchmarks are reported for performance comparisons.

Knowledge Graph



Sign up or login to leave a comment