We develop a general setting for the quantization of linear bosonic and fermionic field theories subject to local gauge invariance and show how standard examples such as linearized Yang-Mills theory and linearized general relativity fit into this framework. Our construction always leads to a well-defined and gauge-invariant quantum field algebra, the centre and representations of this algebra, however, have to be analysed on a case-by-case basis. We discuss an example of a fermionic gauge field theory where the necessary conditions for the existence of Hilbert space representations are not met on any spacetime. On the other hand, we prove that these conditions are met for the Rarita-Schwinger gauge field in linearized pure N=1 supergravity on certain spacetimes, including asymptotically flat spacetimes and classes of spacetimes with compact Cauchy surfaces. We also present an explicit example of a supergravity background on which the Rarita-Schwinger gauge field can not be consistently quantized.