Tracking Issue for os_str_bytes

[epage]

Feature gate: #![feature(os_str_bytes)]

This is a tracking issue for cross-platform access to the underling bytes (&[u8]) for &OsStr by defining it as an unspecified superset of UTF-8.

Assumptions:

• Owned variants of this API are not a blocker for stablization
• Panicking and erroring variants of from_os_str_bytes_unchecked are not a blocker for stablization

Public API

impl OsStr {
    pub unsafe fn from_os_str_bytes_unchecked(bytes: &[u8]) -> &Self;
    pub fn as_os_str_bytes(&self) -> &[u8];
}

impl OsString {
    pub unsafe fn from_os_str_bytes_unchecked(bytes: Vec<u8>) -> Self;
    pub fn into_os_str_bytes(self) -> Vec<u8>;
}

Steps / History

• Implementation: Allow limited access to OsStr bytes #109698, Allow limited access to OsString bytes #113442
• Final comment period (FCP)¹
• Stabilization PR

Unresolved Questions

• What should we refer to this as?
  • Currently os_str_bytes
  • encoded_bytes was another name that was brought up

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[epage]

@RalfJung since you are one of the few to have used this in practice, any feedback? Any thoughts on how well the name works?

[RalfJung]

🤷 no strong opinion on the name. It would have been nice to also have a safe function in the other direction; Miri still needs cfg to convert bytes to OsStr. Having the function return Option is fine as long as it guarantees that it will always succeed on slices produced by as_os_str_bytes and on valid UTF-8. Having only one direction safely feels asymmetrical.

[marcospb19]

About naming, I opened #113106 and it contains this snippet:

let self_len = self.as_os_str().len();
let self_bytes = self.as_os_str().as_os_str_bytes();

Having the type name inside of the method usually leads to repetition, I'd suggest .as_slice() and .from_slice_unchecked since there's no such thing as "os_str_bytes", but I'm not convinced it's better, it reminds me of Vec::as_slice.

Well, assuming that .as_os_str_bytes should deprecate std::os::unix::ffi::OsStrExt::as_bytes anyways, can't we take the breaking change and call this .as_bytes()? If not, I'm also fine with the current name :).

---

I agree with @RalfJung, a from_os_str_bytes -> Option<_> here would be very convenient, compared with the unsafe version.

[epage]

Having the type name inside of the method usually leads to repetition, I'd suggest .as_slice() and .from_slice_unchecked since there's no such thing as "os_str_bytes", but I'm not convinced it's better, it reminds me of Vec::as_slice.

The intention is to specify the encoding within the function name, much like utf8 and utf16 functions have the encoding in their name. The main difference is the encoding is opaque. One idea was as_encoded_bytes.

Well, assuming that .as_os_str_bytes should deprecate std::os::unix::ffi::OsStrExt::as_bytes anyways, can't we take the breaking change and call this `.as_bytes()´? If not, I'm also fine with the current name :).

We can't take breaking changes :). Unnsure if there are workarounds to avoid it being a breaking change.

However, we would want things to be parallel with both directions.

I agree with @RalfJung, a from_os_str_bytes -> Option<_> here would be very convenient, compared with the unsafe version.

The question is if stabilization of what exists should be blocked on this?

First, we'd need the ability to validating WTF-8 encoding which I don't think we have atm.

Second, we'd have to decide on what the error strategy is, whether None or something more like the error from encoding UTF8.

[programmerjake]

Well, assuming that .as_os_str_bytes should deprecate std::os::unix::ffi::OsStrExt::as_bytes anyways, can't we take the breaking change and call this `.as_bytes()´? If not, I'm also fine with the current name :).

We can't take breaking changes :). Unnsure if there are workarounds to avoid it being a breaking change.

is it actually a breaking change if code calls a different function but with the exact same signature and the exact same effect? seems to me that it wouldn't be breaking since code won't behave differently at all.

[marcospb19]

The question is if stabilization of what exists should be blocked on this?

I vote for not blocking because I'd love to see this stabilized, and the checked encoding would certainly delay it.