Recent rapid strides in memory safety tools and hardware have improved software quality and security. While coarse-grained memory safety has improved, achieving memory safety at the granularity of individual objects remains a challenge due to high performance overheads which can be between ~1.7x-2.2x. In this paper, we present a novel idea called Califorms, and associated program observations, to obtain a low overhead security solution for practical, byte-granular memory safety. The idea we build on is called memory blacklisting, which prohibits a program from accessing certain memory regions based on program semantics. State of the art hardware-supported memory blacklisting while much faster than software blacklisting creates memory fragmentation (of the order of few bytes) for each use of the blacklisted location. In this paper, we observe that metadata used for blacklisting can be stored in dead spaces in a program's data memory and that this metadata can be integrated into microarchitecture by changing the cache line format. Using these observations, Califorms based system proposed in this paper reduces the performance overheads of memory safety to ~1.02x-1.16x while providing byte-granular protection and maintaining very low hardware overheads. The low overhead offered by Califorms enables always on, memory safety for small and large objects alike, and the fundamental idea of storing metadata in empty spaces, and microarchitecture can be used for other security and performance applications.