Particles states transforming in one of the infinite spin representations of the Poincar\'e group (as classified by E. Wigner) are consistent with fundamental physical principles, but local fields generating them from the vacuum state cannot exist. While it is known that infinite spin states localized in a spacelike cone are dense in the one-particle space, we show here that the subspace of states localized in any double cone is trivial. This implies that the free field theory associated with infinite spin has no observables localized in bounded regions. In an interacting theory, if the vacuum vector is cyclic for a double cone local algebra, then the theory does not contain infinite spin representations. We also prove that if a Doplicher-Haag-Roberts representation (localized in a double cone) of a local net is covariant under a unitary representation of the Poincar\'e group containing infinite spin, then it has infinite statistics. These results hold under the natural assumption of the Bisognano-Wichmann property, and we give a counter-example (with continuous particle degeneracy) without this property where the conclusions fail. Our results hold true in any spacetime dimension s+1 where infinite spin representations exist, namely s > 1.