The closed-loop performance of model predictive controllers (MPCs) is sensitive to the choice of prediction models, controller formulation, and tuning parameters. However, prediction models are typically optimized for prediction accuracy instead of performance, and MPC tuning is typically done manually to satisfy (probabilistic) constraints. In this work, we demonstrate a general approach for automating the tuning of MPC under uncertainty. In particular, we formulate the automated tuning problem as a constrained black-box optimization problem that can be tackled with derivative-free optimization. We rely on a constrained variant of Bayesian optimization (BO) to solve the MPC tuning problem that can directly handle noisy and expensive-to-evaluate functions. The benefits of the proposed automated tuning approach are demonstrated on a benchmark continuously stirred tank reactor example.