Superconducting circuits are a promising platform for quantum computing, communication, and sensing. Through many years of development these quantum circuits have achieved quality factors of several million even at single photon powers. Most of this work has been based around aluminum circuits operating at 10mK and in the band of 4-8 GHz. Leveraging insights from niobium-based superconducting radio frequency accelerators we have developed superconducting cavities adapted to interact with superconducting qubits with quality factors exceeding 1 billion. Further we have begun to develop cavities and non-linear circuits at mm-wave frequencies near 100GHz. These developments can enable long lived quantum memories, quantum accelerated searches for dark matter, operation of quantum computers at high temperatures with integrated control, and higher speed operation.