You need to look at the dimmer ratings. The cheapest ones are rated at 300W, though you can get them at 600W or 1000W. The most likely result of you exceeding the rating is that the triac within will fry (most likely the dimmer will cease to dim anymore).

However, it's not uncommon for these things to get warm. The whole front plate is metal to try to dissipate some of this. When used in a multigang box, you may have to derate it (read the dimmer instructions).

The dimmer is rated for 600w installed separately or 500w installed in a 2 gang box (my case). So, I believe that part is OK especially since I even reduced to 375w.

Perhaps the temp is normal as you say, but it appears strange to me.

The semiconductor device in the dimmer (a triac) has a small voltage drop when it is conducting.

If the lights are at full intensity and pull 375 watts, the triac is passing ~3.2 amps. If it is dropping just 2 V the power dissipated in the triac is 6 watts.

While not a lot of power it will cause the faceplate (used as a heat sink for the triac) to heat up.