We briefly review two recently developed extensions of the Lorentzian geometry of spacetime and prove that they are in fact closely related. The first is the concept of observer space, which generalizes the space of Lorentzian observers, i.e., future unit timelike vectors, using Cartan geometry. The second is the concept of Finsler spacetimes, which generalizes the Lorentzian metric of general relativity to an observer-dependent Finsler metric. We show that every Finsler spacetime possesses a well-defined observer space that can naturally be equipped with a Cartan geometry. Conversely, we derive conditions under which a Cartan geometry on observer space gives rise to a Finsler spacetime. We further show that these two constructions complement each other. We finally apply our constructions to two gravity theories, MacDowell-Mansouri gravity on observer space and Finsler gravity, and translate their actions from one geometry to the other.