This paper presents a general framework for a refined spectral analysis of a group of isometries acting on a Banach space, which extends the spectral theory of Arveson. The concept of continuous Arveson spectrum is introduced and the corresponding spectral subspace is defined. The absolutely continuous and singular-continuous parts of this spectrum are specified. Conditions are given, in terms of the transposed action of the group of isometries, which guarantee that the pure-point and continuous subspaces span the entire Banach space. In the case of a unitarily implemented group of automorphisms, acting on a $C^*$-algebra, relations between the continuous spectrum of the automorphisms and the spectrum of the implementing group of unitaries are found. The group of spacetime translation automorphisms in quantum field theory is analyzed in detail. In particular, it is shown that the structure of its continuous spectrum is relevant to the problem of existence of (infra-)particles in a given theory.