Well on all benching exercises you use the lats to assist with the movement. The wider you grip the bar, the more lats you use. Using dumbells, you may be getting wide at some point in the movement and using the lats more than you realize. Guess where the lats attach??

Right where you are describing your pain.
I get the same thing when I bench and I use a bar exclusively to keep my form correct. Besides, I'm afraid of the dumbells - I dumped a 70lb adjustable on my chest pointy end first once and broke two ribs.