We propose computationally inexpensive and efficient estimators for multivariate stochastic volatility (MSV) mod- els with cross-dependence, Granger causality, and higher-order persistence in latent volatilities. The proposed class of estimators is based on a small set of moment equations derived from the weak VARMA representations of MSV models. Except for the cross-dependence parameters, closed-form expressions are obtained for the remaining pa- rameters, so that no numerical optimization is required. To improve the stability and efficiency of the volatility- persistence parameter estimates, we develop shrinkage-type VARMA estimators based on averaging different simple estimates (winsorization) and matrix-variate regressions (MVR). We also derive the asymptotic distribution of these estimators. Due to their computational simplicity, VARMA estimators make reliable — possibly exact — simulation- based inference tractable through Monte Carlo test techniques. We formulate volatility commonality as a testable common-C-space restriction on the MSV transition matrices, and develop finite-sample Monte Carlo tests for this hypothesis. Simulation results show that the proposed shrinkage-type VARMA-MVR estimators compare favorably with Bayesian and QML benchmarks in terms of bias, RMSE, and computational cost. They also remain feasible in very high dimensions, and, using auxiliary particle filtering, deliver accurate latent-state recovery and covariance forecasts even under model misspecification. In empirical applications to stock returns, we find that unrestricted higher-order MSV models perform well for covariance forecasting and portfolio building, while the data provide little support for an exact low-dimensional common dynamic subspace.