The superselection sectors of two classes of scalar bilocal quantum fields in $D\ge 4$ dimensions are explicitly determined by working out the constraints imposed by unitarity. The resulting classification in terms of the dual of the respective gauge groups $U(N)$ and $O(N)$ confirms the expectations based on general results obtained in the framework of local nets in algebraic quantum field theory, but the approach using standard Lie algebra methods rather than abstract duality theory is complementary. The result indicates that one does not lose interesting models if one postulates the absence of scalar fields of dimension $D-2$ in models with global conformal invariance. Another remarkable outcome is the observation that, with an appropriate choice of the Hamiltonian, a Lie algebra embedded into the associative algebra of observables completely fixes the representation theory.