explicitly keep the door open for some but not all subobject provenance

[RalfJung]

Miri has enforced fairly strict subobject provenance by default since ~forever, but the Reference never explicitly called this out as UB. Let's fix that. This is the conservative choice; we document this as UB now and maybe lift the UB restriction again in the future.

This PR includes two commitments that @rust-lang/opsem (and maybe @rust-lang/lang) should FCP:

• While we generally keep the door open for subobject provenance, we explicitly say that there is no subobject provenance within arrays/slices. This is based on the observation that violations of such subobject provenance are the major kind of Stacked Borrows UB that Miri finds. I'd like to make Miri stop report this as UB, and for this I'd like to be sure that indeed we don't want to make this UB.
• While we generally keep the door open for "discriminant protection", we do explicitly say that it is okay to temporarily change the discriminant of an enum via a field reference, and then change it back. This is needed to make code like this sound, which came up multiple times in questions recently (1, 2).

I proposed for opsem to FCP these two choices in:

• No subobject provenance for arrays/slices unsafe-code-guidelines#622
• Can references inside an enum transiently set an invalid enum discriminant? unsafe-code-guidelines#621

This PR should only be merged once both of those FCP completed.

[joshlf]

How does this play with transmutes? Consider:

#[repr(C)]
struct Pair {
    first: [usize; 4],
    second: [usize; 4],
}

let pair = &Pair { ... };
let arr: &[usize; 8] = unsafe { &*(pair as *const Pair) };
let i0: *const usize = &arr[0];
let i7 = unsafe { i0.add(7) };
let i7 = unsafe { &*i7 };

This goes from:

• &Pair to &[usize; 8] (a sound transmute)
• ...to a raw pointer to the 0th element
• ...to a raw pointer to the 7th element
• ...to a reference to the 7th element

Presumably if we skipped the &Pair -> &[usize; 8] step and instead constructed i0 from &pair.first[0], this would be unsound because it would entail jumping between first and second.

Two questions:

• Am I correct that the code as written is sound, and that if we skipped &Pair -> &[usize; 8], it would not be sound?
• If so, what accounts for the differing soundness between these two examples?

View changes since the review

[RalfJung]

The narrowing is attached to field projections. So as long as you don't do field projections, you retain access to the full allocation. Your example code does not do field projections, so it is fine.

Presumably if we skipped the &Pair -> &[usize; 8] step and instead constructed i0 from &pair.first[0], this would be unsound

Indeed, now there is a field projection that narrows the provenance.

[joshlf]

Can we clarify what "this exception also applies for nested arrays" means? I presume it means that you can go from e.g. x[0][0] to x[1][1] (where x: [[usize; 2]; 2])? What about from x[0] to x[1][1] or from x[0][0] to x[1]?

View changes since the review

[RalfJung]

This is all just meant to say "as long as you're not projecting to a field, you're not restricted to a subobject". I am not sure what the best way to say that is.

[joshlf]

What does "as in the previous paragraph" refer to?

View changes since the review

[RalfJung]

Writing the tag may violate subobject provenance. So temporarily changing the discriminant is only allowed if that discriminant actually lies within the memory range this pointer is allowed to mutate.

[joshlf]

What if changing the discriminant has the effect of just doing a transmute? E.g.:

#[repr(u8)]
enum Zero { Zero = 0 }

#[repr(u8)]
enum One { One = 1 }

enum Bit {
    Zero(Zero),
    One(One),
}

If Rust chooses to niche-optimize Bit so that there's no explicit discriminant, then given *mut Zero pointing to the only field of Bit::Zero, you could overwrite it with One::One and you'd effectively have transmuted the entire Bit to Bit::One(One::One). Is this sound even if the discriminant is not restored?

View changes since the review

[RalfJung]

Is this sound even if the discriminant is not restored?

No. That's the kind of code @digama0 would like to disallow. I don't necessarily agree, but want to make some progress without having to resolve the question entirely, so this PR does not intend to allow it.

[joshlf]

We often say that lifetimes are not relevant to opsem. What does the term "lifetime" mean in this context? Is this about crossing an API boundary (i.e., this is really a safety thing about what you're allowed to assume when an enum reference/pointer crosses an API boundary), or is this about SB/TB, or something else?

View changes since the review

[RalfJung]

I use "lifetime" here for the same reason that it is already used one bullet point up the list: this isn't the final opsem, it is an approximation to give users some guidance. We don't want to specify when exactly provenance gets invalidated due to conflicting accesses. In SB/TB, this happens some time after the lifetime ends. But if we conservatively say that it happens exactly when the lifetime ends, that's a useful approximation.

This is already the wording we use here, so I followed the same approach for this new item in the list.

[RalfJung]

There's more pushback than I expected for this one so it's not really ready for lang / lang-docs discussion yet I think.

@rustbot label -I-lang-docs-nominated