This page is a snapshot from the LWG issues list, see the Library Active Issues List for more information and the meaning of C++20 status.

### 3328. Clarify that `std::string` is not good for UTF-8

Section: D.25 [depr.fs.path.factory] Status: C++20 Submitter: The Netherlands Opened: 2019-11-07 Last modified: 2021-02-25

Priority: 0

View all issues with C++20 status.

Discussion:

Addresses NL 375

Example in deprecated section implies that `std::string` is the type to use for utf8 strings.

> [Example: A string is to be read from a database that is encoded in UTF-8, and used to create a directory using the native encoding for filenames:

>     namespace fs = std::filesystem;
>     std::string utf8_string = read_utf8_data();
>     fs::create_directory(fs::u8path(utf8_string));

Proposed change:

Add clarification that `std::string` is the wrong type for utf8 strings

Jeff Garland:

SG16 in Belfast: Recommend to accept with a modification to update the example in D.25 [depr.fs.path.factory] p4 to state that `std::u8string` should be preferred for UTF-8 data.

Rationale: The example code is representative of historic use of `std::filesystem::u8path` and should not be changed to use `std::u8string`. The recommended change is to a non-normative example and may therefore be considered editorial.

Previous resolution [SUPERSEDED]:

> This wording is relative to N4835.

> 1. Modify D.25 [depr.fs.path.factory] as indicated:

>    > -4- [Example: A string is to be read from a database that is encoded in UTF-8, and used to create a directory using the native encoding for filenames:

>    > >     namespace fs = std::filesystem;
>    > >     std::string utf8_string = read_utf8_data();
>    > >     fs::create_directory(fs::u8path(utf8_string));

>    > For POSIX-based operating systems with the native narrow encoding set to UTF-8, no encoding or type conversion occurs.

>    > For POSIX-based operating systems with the native narrow encoding not set to UTF-8, a conversion to UTF-32 occurs, followed by a conversion to the current native narrow encoding. Some Unicode characters may have no native character set representation.

>    > For Windows-based operating systems a conversion from UTF-8 to UTF-16 occurs. — end example]

>    > [Note: The example above is representative of historic use of `filesystem` `u8path`. New code should use `std::u8string` in place of `std::string`. — end note]

LWG Belfast Friday Morning

Requested changes:

- Historic => historical.
- Add missing :: before u8path.
- Remove ISO rules forbidden 'should' in a note.
- Use language describing why new code should use the u8string constructor rather than preaching that new code should do something.

Billy O'Neal provides updated wording.

[2020-02 Moved to Immediate on Tuesday in Prague.]

Proposed resolution:

This wording is relative to N4835.

1. Modify D.25 [depr.fs.path.factory] as indicated:

   > -4- [Example: A string is to be read from a database that is encoded in UTF-8, and used to create a directory using the native encoding for filenames:

   > >     namespace fs = std::filesystem;
   > >     std::string utf8_string = read_utf8_data();
   > >     fs::create_directory(fs::u8path(utf8_string));

   > For POSIX-based operating systems with the native narrow encoding set to UTF-8, no encoding or type conversion occurs.

   > For POSIX-based operating systems with the native narrow encoding not set to UTF-8, a conversion to UTF-32 occurs, followed by a conversion to the current native narrow encoding. Some Unicode characters may have no native character set representation.

   > For Windows-based operating systems a conversion from UTF-8 to UTF-16 occurs. — end example]

   > [Note: The example above is representative of a historical use of `filesystem::u8path`. Passing a `std::u8string` to `path`'s constructor is preferred for an indication of UTF-8 encoding more consistent with `path`'s handling of other encodings. — end note]