Since items with a gems are usually better then items without it if you gemmed it correctly (only with best gems available), every gem counts. However, that bonus is very small.
On the other side, if item is gemmed with cheap or wrong gems, it will have worse score than item without gems.
