Reference | How to Detect File Encoding on Windows, Linux and Mac - Support and Troubleshooting

Reference | How to Detect File Encoding on Windows, Linux and Mac File encoding refers to the way text is represented in a file.

Tools and commands are available on Windows , Linux , and Mac to help determine file encoding.

Detecting File Encoding on Windows

Using Notepad Notepad provides a user-friendly way to detect file encoding.

Steps :

Open the file in Notepad . Go to the View menu at the top of the window, make sure " Status Bar " is selected. Notepad shows the detected encoding (e.g., UTF-8, ANSI) in the status bar located at the bottom of the window. Using 'CertUtil' command CertUtil is a built-in tool in Windows that can dump a file's contents in hexadecimal. It helps you detect the BOM, which indicates certain encodings.

Steps :

Open a terminal . Run the following command: certutil -dump C:\path\to\your\file.txt

The first few bytes in the output will help you identify if there's a BOM present. For example:

UTF-8: EF BB BF UTF-16 (BE): FE FF UTF-16 (LE): FF FE UTF-32 (BE): 00 00 FE FF UTF-32 (LE): FF FE 00 00

*In cases where a file's encoding is not explicitly specified (e.g., no BOM is present), determining the encoding can be ambiguous. Common encodings like UTF-8 (without BOM), ASCII, and ANSI can look identical for certain text files, especially those containing only basic characters. In such situations, PowerShell and most text editors will make an assumption about the encoding based on system defaults or the content itself, but this can sometimes lead to incorrect interpretations, especially with non-ASCII characters.

Using 'PowerShell' There is no way for PowerShell to automatically determine the file encoding.

However, you can load the content with different encodings.

Steps :

Open a PowerShell . Run the following command: Get-Content -Path "C:\path\to\your\file.txt" -Encoding utf8

The acceptable values for encoding parameter are as follows:

ascii: Uses the encoding for the ASCII (7-bit) character set. ansi: Uses the encoding for the current culture's ANSI code page. *This option was added in PowerShell 7.4.

bigendianunicode: Encodes in UTF-16 format using the big-endian byte order. bigendianutf32: Encodes in UTF-32 format using the big-endian byte order. oem: Uses the default encoding for MS-DOS and console programs. unicode: Encodes in UTF-16 format using the little-endian byte order. utf7: Encodes in UTF-7 format. utf8: Encodes in UTF-8 format. utf8BOM: Encodes in UTF-8 format with Byte Order Mark (BOM) utf8NoBOM: Encodes in UTF-8 format without Byte Order Mark (BOM) utf32: Encodes in UTF-32 format.

If you encounter ambiguity in encoding detection, you can try reading the file using different encodings manually by specifying them with the '-Encoding' parameter in 'Get-Content'. For instance, try reading the file with 'utf8', 'ascii', or 'unicode' to see which encoding renders the text correctly.

Detecting File Encoding on Linux

Using 'file' Command The 'file' command is one of the easiest and most popular ways to check a file's encoding.

Steps :

Open a terminal . Run the following command: file -i /PATH/file.txt

This command will return the file type and encoding.

For example : file.txt: text/plain; charset=utf-8

Using Hex Editor Converting the content of a file into a hexadecimal dump.

Steps :

Open a terminal . Select and run one of the following commands: xxd filename.txt | head -n 1 hexdump -C filename.txt | head -n 1 *head -n 1 limits the output to just the first line

The first few bytes in the output will help you identify if there's a BOM present. For example:

UTF-8: EF BB BF UTF-16 (BE): FE FF UTF-16 (LE): FF FE UTF-32 (BE): 00 00 FE FF UTF-32 (LE): FF FE 00 00 If you encounter ambiguity in encoding detection, you can try reading the file using different encodings manually.

For instance, try reading the file with 'utf8', or 'ascii' to see which encoding renders the text correctly: iconv -f UTF-8 -t UTF-8 filename.txt

Detecting File Encoding on Mac

Using 'file' Command The 'file' command is one of the easiest and most popular ways to check a file's encoding.

Steps :

Open a terminal . Run the following command: file -I /PATH/file.txt

This command will return the file type and encoding.

For example : file.txt: text/plain; charset=us-ascii

Using Hex Editor Converting the content of a file into a hexadecimal dump.

Steps :

Open a terminal . Select and run one of the following commands: xxd filename.txt | head -n 1 hexdump -C filename.txt | head -n 1 *head -n 1 limits the output to just the first line

The first few bytes in the output will help you identify if there's a BOM present. For example:

UTF-8: EF BB BF UTF-16 (BE): FE FF UTF-16 (LE): FF FE UTF-32 (BE): 00 00 FE FF UTF-32 (LE): FF FE 00 00 If you encounter ambiguity in encoding detection, you can try reading the file using different encodings manually.

For instance, try reading the file with 'utf8', or 'ascii' to see which encoding renders the text correctly: iconv -f UTF-8 -t UTF-8 filename.txt

Version Published Summary of Changes 1.0 dd-mmm-yyyy Initial version 2.0 --- Provide a summary of the updates made to each additional version of this article.