Traditional methods for performance appraisal are not suitable for agile fast-paced software companies. This has been a realization in the software industry since the early adoption of agile methodologies. Nonetheless, software companies are still struggling to find a practical solution that fits the highly dynamic nature of their teams. In particular, high-growth companies, scaleups, need to be creative at how they approach performance appraisals. In this talk, we introduce automated review dashboards to support a seamless appraisal process and continuous visibility of software engineers' performance. The proposed dashboards leverage structure from existing growth frameworks while automating the aggregation of relevant qualitative and quantitative performance metrics. We reflect on our experience using the dashboards at Robusta, a medium-sized software scaleup company. The dashboards enabled a team of four managers to provide timely feedback to 56 engineers with a base for continuous visibility. We explore the design of the dashboards, the customizable metrics and the overall review experience from the perspectives of different stakeholders. We conclude with the lessons learned and practical advice for scaleups facing the same challenge.