The number on the turqouise ribbon next to an item in your Inventory shows the number of these items that have been added recently. New items have already been included in your total item count.
Once you scroll through all the items with turqouise ribbons, the exclamation point will disappear from the Inventory category.