Let $M$ be a factor with separable predual and $G$ a compact group of automorphisms of $M$ whose action is minimal, i.e. $M^{G^\prime}\cap M = C$, where $M^G$ denotes the $G$-fixed point subalgebra. Then every intemediate von Neumann algebra $M^G\subset N\subset M$ has the form $N=M^H$ for some closed subgroup $H$ of $G$. An extension of this result to the case of actions of compact Kac algebras on factors is also presented. No assumptions are made on the existence of a normal conditional expectation onto $N$.