A perfect data set does not have to be as elusive as nirvana. If you shudder at the memories of the Y2K and other "big bang" events then why not ask Openstrike to help with wholesale changes to your data sets?
If you need to make automated changes to your data, we can provide the service you are after, whether you have a few hundred records or a few hundred million. We are able to provide this service for JSON, XML, CSV and fixed-fieldwidth files making it independent from any particular database engine.
Openstrike can also provide manipulation services for data sets which require sorting, field swapping, field concatenation, or other conversion processes. If you have any specific or unusual data management requirements, we will be happy to discuss them with you in advance.