Rolling for defense?

Hello everyone,

I like the idea of a game where practically any action where the outcome is uncertain is handled with a roll, and when that roll is made against an opponent, the opponent needs to make a contesting roll. When a character attempts something requiring a feat of strength, such as lifting something heavy, they must roll a d20 and add their STR. When an attempt like that is made against an opponent, like trying to restrain them, both roll a d20 and add their STR. Pretty standard, nothing crazy.

However, I want to carry this system over into attack and defense rolls. The attacker rolls a d20 and adds the appropriate ability score, while the defender rolls a d20 and adds their defense score. This is similar to an optional rule in Knave 1e. Because I am hacking Knave 2e, I have item slots. Given that, a single piece of armor occupying an item slot (max of 7) would count as a point towards the character's defense score. If you want a more evasive character, each empty item slot up to half of their DEX score (so max of 5) is added to their defense score. So to avoid getting hit, you roll a d20 + your total defense score and hope to roll higher than the attacker’s attack roll.

What this would mean is that if you have 10 DEX and at least 2 CON, you could have a +12 defense score, but all of your slots would be occupied. I think this is balanced, partially because of the item slot system, because an attackers to hit bonus can be as high as a +10, and because the defender can still roll low and the attacker roll high. However, maybe I’m wrong and it’s completely broken. I have a really hard time looking at things and seeing how it could work out. Please poke holes in this, I'd love some feedback.

ADDITIONALLY. I understand that this will slow down combat, but half the fun of the game is rolling dice, and this means both parties get to roll more dice. I have always found checks made to grapple or restrain far more engaging because both parties get to roll dice and see who wins, as opposed to rolling a die and asking if it meets or exceeds a pre-existent value. However, this means nothing if the system is fundamentally flawed, so I would really appreciate feedback.