Silverlight DataServiceQuery $expand caching fields

I have a DataServiceQuery in Silverlight selecting from ServiceActivity and expanding on Contacts to return contact fields.

I've noticed that if I'm in my application and viewing an SA it correctly shows the contact field. However, if I go into CRM and change this Contact field, my Silverlight application does not update the Contact field.

For grins, I added the same field to the SA and did the same test. It correctly changed the field.

For some reason, the fields from the $expand are not updating after I modify them in CRM unless I close my browser and come back to it. Is there anything I can do in my Silverlight application to make sure it "refreshes" and gets the new value?

Thanks