We introduce and study the crossing map, a closed linear map acting on operators on the tensor square of a given Hilbert space that is inspired by the crossing property of quantum field theory. This map turns out to be closely connected to Tomita--Takesaki modular theory. In particular, crossing symmetric operators, namely those operators that are mapped to their adjoints by the crossing map, define endomorphisms of standard subspaces. Conversely, such endomorphisms can be integrated to crossing symmetric operators. We also investigate the relation between crossing symmetry and natural compatibility conditions with respect to unitary representations of certain symmetry groups, and furthermore introduce a generalized crossing map defined by a real object in an abstract $C^*$-tensor category, not necessarily consisting of Hilbert spaces and linear maps. This latter crossing map turns out to be closely related to the (unshaded, finite-index) subfactor theoretical Fourier transform. Lastly, we provide families of solutions of the crossing symmetry equation, solving in addition the categorical Yang--Baxter equation, associated with an arbitrary Q-system.