Passive are interesting because they offer others mecanisms too.
CD is a good for plenty of them: for when and on hit it’s normal to have CD. It would be too porwerfull, unless you make them weak and so no interesting.
Tremor is a good exemple: it stun ennemy around you when hit. whitout CD it is overpower. If it is only chance, it’s too random.
Powerfull first blow each 4s allow you to burst dangerous ennemy andf give you another mecanism to get in your mind. It’s only if you want it and it could transform your way of playing. It’s great.
On crit doesn’t need CD I think, because you have to build around crit to augment chance to proc the passive, so it’s already a “trade off”.
So CD are OK, if of course the power of the passive and the duration of the passive are well balanced.