Skip to Main Content
SEI Idea Portal
Status Submitted
Created by Sue Waters
Created on Jan 8, 2025

Need Conversion Tool for correcting SQL Collation

In countries outside the US, the collation of the sql server is based on windows locale and the default during a sql installation for most non-US countries particularly in Europe is Latin1 versus the SQL Latin1 which is the official collation of the EA. In order to bring in the countries correct collation after data is added to an archive with the incorrect collation, the data must be exported out to a new database with the correct SQL 1 collation. The customer would be down during that period of time. On our larger database of 1 TB or even several 100 gigabytes the customer would be down for a significant period of time in the neighborhood of several days.

Microsoft warns if the data is not exported to a new database then there is a possibility of "silent data loss". It is Microsoft's recommended best practice.

Also, the incorrect collation can cause issues with SQL Performance and even affect the information returned in a select query with "like" where more than the expected result is returned.

I think the work already done on the EA's 64bit conversion software could be leveraged in this case to allow the data to be exported into the correct collation while allowing the customer to be up and running during that period. Minimizing downtime. It would have to be altered to allow for the other non-archive databases in the EA.


Links to MS articles for the collation issue:

https://learn.microsoft.com/en-us/sql/relational-databases/collations/collation-and-unicode-support?view=sql-server-ver16#Server-level-collations

https://learn.microsoft.com/en-us/sql/relational-databases/collations/set-or-change-the-database-collation?view=sql-server-ver16&tabs=ssms

Non-MS articles:

https://www.sqlservercentral.com/blogs/revised-difference-between-collation-sql_latin1_general_cp1_ci_as-and-latin1_general_ci_as

https://stackoverflow.com/questions/73652503/sql-server-collation-default-for-us-english-best-practice