Transforms: scope duplicate-user fusion for layout propagation - #22374
Transforms: scope duplicate-user fusion for layout propagation#22374rascani wants to merge 4 commits into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22374
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 871143b with merge base 684d4bd ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This patch does at least 5 different things, would it be possible to split it or at least document it a bit to help the review? |
8518bd7 to
9576d24
Compare
Sorry about that. It does a lot, but I'd prefer not to split it because its all related to making a safe decision about whether two nodes can share the same output. I've updated the summary to (hopefully) better explain what it is doing and also included a suggested review order. |
FuseDuplicateUsersPass previously identified duplicates from operator and arguments alone. That could merge nondeterministic or mutating calls, distinct literal tensors, values with backend-specific metadata, or independently returned tensors. Restrict fusion to schema-known pure non-aliasing operators, respect intervening effects and downstream mutations, and preserve output identity by default. Arm includes quantization metadata in the signature and opts into output aliasing only where EnsureUniqueOutputNodesPass repairs it before serialization. AI-assisted: Codex.
Build the duplicate-fusion semantic key through the existing FoldAndAnnotateQParamsPass accessors while retaining the empty-metadata behavior used by float graphs. AI-assisted: Codex.
Limit horizontal duplicate fusion to the view and permute transforms owned by the propagation pass. Thread the existing semantic-key policy through that call site so Arm transformations with different folded quantization parameters remain distinct. AI-assisted: Codex.
a8b0040 to
197b01c
Compare
|
The operator is nondeterministic / equal-looking literal tensors are different objects. -> Seems reasonable but how often are literal tensors and non deterministic ops users of previous ops? Could the relevant ops be listed for simplicity? Operator is mutable, or returns an alias. / An effect occurs between the two calls. / One result, or an alias of it, is later mutated. -> All of these are about aliasing and muteability, do we handle such cases usually and would this not be an issue for all decompositions in that case? Also, the code is a bit complex, could it be simplified by just doing one pass of marking nodes which are mutated by another node and directly bailing on them or is there any case which is missed? Both results are independently returned. -> Could this be simplified by just not fusing nodes which are inputs to the output node or am I missing some subtlety? Backend metadata changes the meaning of the operation. -> OK |
Keep only the target allowlist and backend semantic key needed by layout propagation. Restore the existing generic fusion behavior and Arm cleanup policy, while restricting propagation fusion to non-aliasing layout-copy operators. AI-assisted: Codex.
Summary
Layout propagation uses
FuseDuplicateUsersPassto merge duplicate view and permute users exposed while transforms move through a graph.This PR adds two optional controls:
allowed_targetslimits fusion to the layout-copy operators owned by propagation.semantic_keyincludes backend metadata not represented in FX arguments.Arm uses the semantic key for input and output qparams, preventing otherwise-identical layout copies with different
quantization semantics from being merged.
Existing callers retain their current behavior. This PR does not change the generic pass’s handling of mutation, aliasing, effects, literals, or graph outputs. Arm’s existing
RESCALEexclusion and output cleanup also remain unchanged.Review guide
fuse_duplicate_users_pass.py: the two optional configuration points.propagate_view_copy_permute_pass.py: restricting fusion to layout copies.AI-assisted: Codex.
Stack created with GitHub Stacks CLI • Give Feedback 💬
cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell