I don't know of any real data out there that says changing the IP address of a site will negatively affect you as far as Google is concerned. However, make sure that:

1. The new server is just as fast and well-optimized as the old server
2. The new host/server is still in the same country/location as your targeted audience (i.e. don't host a US-Based company in Zimbabwe).