Let $x$ be a CM point in the moduli space $\mathcal{A}_g(\mathbb{C})$ of principally
polarized complex abelian varieties of genus $g$, corresponding to an
Abelian variety $A$ with complex multiplication by a ring $R$. Edixhoven
conjectured that the size of the Galois orbit of x should grow at least
like a power of the discriminant ${\rm Disc}(R)$ of $R$. For $g=1$, this reduces to the
classical Brauer-Siegel theorem. A positive answer to this conjecture
would be very useful in proving the Andr\'e-Oort conjecture unconditionally.
We will present a proof of the conjectured lower bounds in some special
cases, including $g\le 6$. Along the way we derive transfer principles for
torsion in class groups of different fields which may be interesting in
their own right.