This has bugged me for ages and I've finally spent 10 minutes finding the solution to it.

I develop lots of custom controls which I will never place on a form at design time, their add-on components that are loaded at runtime and docked in conditionally.  This ended up with Solution bloat meaning every time I first went into a design view of a form Visual Studio went into a fit building up a complete toolbox of things I'd never need.

One simple attribute was all that was needed to stop them being populated in future;

<System.ComponentModel.DesignTimeVisible(False)> _
Public Class MyUserControl ....

Hopefully someone else will find this useful for the future.