All skills have an added damage effectiveness, which is a multiplier that is applied to all damage from stats like "+5 melee fire damage".
Skills with large mana costs or cooldowns typically have greater than 100% damage effectiveness, while some skills that hit rapidly have less than 100%.
For example, Elemental Nova has a damage effectiveness of 100%, while Meteor has a damage effectiveness of 900%.
If you had "+10 spell fire damage" from your passives, 10 fire damage would be added to Elemental Nova, while 90 fire damage would be added to Meteor. You can view the damage effectiveness of skills by holding ALT while hovering on them.
Damage effectiveness does not affect base damage.