As of right now I have been primarily playing characters that can summon, and I find the mechanic of just scaling physical and generic minion damage a bit of a pablum for scaling damage.
I am sure this is something that will be implemented at a later point in some iteration, but I feel that there should be means to convert damage from other item modifiers to minion damage and vice versa. Additionally, this suggestion can apply to defensive stats as well.
At the moment it feels like items that give me damage modifiers to elemental sources or directly affect my character’s physical damage are wasted affixes on my gear as a summoner. It could make these affixes meaningful for a cookie cutter type build, or jack of trades, if there were either: special affixes you could balance your other gear pieces with that convert a percentage of one damage source to another or give further options on the passives tree or the skill specialization trees that would allow your summons to take advantage of the other affixes on your gear and vice versa.