Add passive command-line option defaults - #10895
Merged
Amaury Levé (Evangelink) merged 5 commits intoSep 1, 2026
Merged
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f86614b5-b40f-415d-8f94-a3290fbe7154
This comment has been minimized.
This comment has been minimized.
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Note
🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.
| # | Dimension | Verdict |
|---|---|---|
| 5 | Performance & Allocations | 🟡 1 NIT |
| 15 | Code Structure & Simplification | 🟡 1 NIT |
✅ 20/22 dimensions clean.
Summary: This is a well-structured, thorough PR that adds passive commandLineOptionDefaults support to testconfig.json with clean separation between active options and defaults. The design is sound — defaults never activate features, explicit values take precedence, and the MSBuild integration via TestingPlatformCommandLineOptionDefault items is ergonomic.
Key positives:
- Public API surface is minimal (one extension method + one internal interface)
PublicAPI.Unshipped.txtandInternalAPI.Unshipped.txtare both updated correctly- The
#if NETCOREAPP/ Jsonite split keeps the netstandard2.0 MSBuild task self-contained - Validation (unknown options, arity, bootstrap-only, per-arg) is consistently extended to the new section
- Test coverage is comprehensive: unit tests for merge logic, defaults-don't-activate, explicit-wins, error cases; plus acceptance tests for both MSBuild generation and TRX runtime behavior
- Localization resources and xlf files are properly updated
- JSON schema updated to match
Minor observations (inline):
- The MSBuild
Inputssentinels for option defaults may disable incremental build for this target — worth monitoring -
CommandLineOptionsProxydelegates to the extension method which re-checks the interface cast (functionally correct, slightly indirect)
Contributor
🧪 Expert test review — PR #10895
All reviewed tests directly exercise the passive command-line option defaults feature added in this PR (MSBuild This advisory comment was generated automatically. Grades are heuristic and informational — they do not block merging. Suggestions on the Files changed tab can be applied with one click. Re-run with
|
Contributor
There was a problem hiding this comment.
Copilot review overview
Review tier: Balanced
Findings: 3
New issues introduced by this change (3)
| Severity | Finding |
|---|---|
 |
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets — These transformed values are not real files, so MSBuild cannot use their timestamps as stable… |
|
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/ConfigurationFileTask.cs — This shipped fallback is not as strict as the System.Text.Json branch. The package always loads the… |
|
src/Platform/Microsoft.Testing.Platform/Configurations/JsonConfigurationProvider.CommandLineOptions.cs — Reusing this enumerator also reuses its permissive empty-container behavior: empty arrays are… |
What changed in this PR
Adds passive command-line option defaults across MTP configuration, MSBuild integration, and report filename handling, addressing #6648.
Changes:
- Adds validated defaults with explicit-option precedence.
- Projects MSBuild items into generated
testconfig.json. - Applies defaults to report filenames and adds documentation/tests.
| File | Description |
|---|---|
test/UnitTests/Microsoft.Testing.Platform.UnitTests/Configuration/JsonCommandLineOptionsTests.cs |
Tests parsing and validation. |
test/UnitTests/Microsoft.Testing.Platform.UnitTests/CommandLine/CommandLineHandlerTests.cs |
Tests passive lookup and precedence. |
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/MSBuildTests.cs |
Tests configuration merging. |
test/UnitTests/Microsoft.Testing.Platform.MSBuild.UnitTests/InvokeTestingPlatformTaskTests.cs |
Updates filesystem stub. |
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/TrxTests.cs |
Tests TRX defaults and precedence. |
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/MSBuildTests.ConfigurationFile.cs |
Tests MSBuild-generated defaults. |
src/Platform/SharedExtensionHelpers/ReportEngineBase.cs |
Applies defaults to shared reporters. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf |
Updates Traditional Chinese resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf |
Updates Simplified Chinese resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf |
Updates Turkish resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf |
Updates Russian resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf |
Updates Portuguese resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf |
Updates Polish resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf |
Updates Korean resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf |
Updates Japanese resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf |
Updates Italian resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf |
Updates French resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf |
Updates Spanish resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf |
Updates German resources. |
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf |
Updates Czech resources. |
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx |
Adds defaults-specific diagnostics. |
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt |
Tracks the new public extension. |
src/Platform/Microsoft.Testing.Platform/InternalAPI/InternalAPI.Unshipped.txt |
Tracks internal API changes. |
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostBuilder.CommonServices.cs |
Loads and validates defaults. |
src/Platform/Microsoft.Testing.Platform/Configurations/PlatformConfigurationConstants.cs |
Defines the defaults section name. |
src/Platform/Microsoft.Testing.Platform/Configurations/JsonConfigurationProvider.CommandLineOptions.cs |
Parses default entries. |
src/Platform/Microsoft.Testing.Platform/Configurations/JsonCommandLineOptionEntry.cs |
Generalizes entry documentation. |
src/Platform/Microsoft.Testing.Platform/Configurations/ConfigurationExtensions.cs |
Implements default lookup and precedence. |
src/Platform/Microsoft.Testing.Platform/Configurations/AggregatedConfiguration.cs |
Resolves defaults across providers. |
src/Platform/Microsoft.Testing.Platform/CommandLine/ICommandLineOptions.cs |
Exposes passive default lookup. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.UnknownAndBootstrapValidation.cs |
Validates unknown/bootstrap defaults. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.cs |
Integrates default validation. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.ArityValidation.cs |
Validates default arity. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsValidator.ArgumentAndConfigurationValidation.cs |
Validates default arguments. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineOptionsProxy.cs |
Forwards default lookup. |
src/Platform/Microsoft.Testing.Platform/CommandLine/CommandLineHandler.cs |
Implements runtime default lookup. |
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/ConfigurationFileTask.cs |
Merges defaults into JSON. |
src/Platform/Microsoft.Testing.Platform.MSBuild/PACKAGE.md |
Documents authoring surfaces. |
src/Platform/Microsoft.Testing.Platform.MSBuild/Microsoft.Testing.Platform.MSBuild.csproj |
Embeds Jsonite for down-level tasks. |
src/Platform/Microsoft.Testing.Platform.MSBuild/IFileSystem.cs |
Adds configuration-file reading. |
src/Platform/Microsoft.Testing.Platform.MSBuild/FileSystem.cs |
Implements file reading. |
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets |
Passes defaults into generation. |
src/Platform/Microsoft.Testing.Extensions.TrxReport/TrxReportEngine.cs |
Applies passive TRX filename defaults. |
docs/testconfig.schema.md |
Documents schema coverage. |
docs/testconfig.schema.json |
Defines defaults schema. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Amaury Levé (Evangelink)
added
the
state/needs-review
Petr Pokorny (0101)
approved these changes
Sep 1, 2026
Amaury Levé (Evangelink)
enabled auto-merge (squash)
September 1, 2026 09:07
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f86614b5-b40f-415d-8f94-a3290fbe7154
Copilot started reviewing on behalf of
Amaury Levé (Evangelink)
September 1, 2026 09:32
View session
This comment has been minimized.
This comment has been minimized.
Contributor
🧪 Expert test review — PR #10895Reviewed the 11 new test methods across 5 changed test files (integration and unit) covering the new passive
This advisory comment was generated automatically. Grades are heuristic and informational -- they do not block merging. Re-run with
|
Contributor
There was a problem hiding this comment.
Copilot review overview
Review tier: Balanced
Findings: 2
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
|
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets — When this item list transitions from non-empty to empty while a source testconfig.json still… |
|
src/Platform/SharedExtensionHelpers/ReportEngineBase.cs — TryGetOptionArgumentListOrDefault returns true for a passive fallback too, so wasExplicit and… |
Issues resolved since last review (3)
| Severity | Finding |
|---|---|
|
src/Platform/Microsoft.Testing.Platform/Configurations/JsonConfigurationProvider.CommandLineOptions.cs — Reusing this enumerator also reuses its permissive empty-container behavior: empty arrays are… View resolved comment |
|
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/ConfigurationFileTask.cs — This shipped fallback is not as strict as the System.Text.Json branch. The package always loads the… View resolved comment |
|
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets — These transformed values are not real files, so MSBuild cannot use their timestamps as stable… View resolved comment |
Suppressed comments (1)
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets:58
- The cache hashes only the default items, while the source path is included as an input only when it currently exists. If a project with defaults deletes
testconfig.json, the item hash is unchanged and the deleted source disappears fromInputs; therefore the core target can be skipped and its output retains all settings from the deleted source file. Include the source-existence state in the cache hash so adding/removing the file invalidates the generated configuration.
<_GenerateTestingPlatformConfigurationFileInputsToHash Include="@(TestingPlatformCommandLineOptionDefault->'%(Identity)=%(Value)')" />
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f86614b5-b40f-415d-8f94-a3290fbe7154
Copilot started reviewing on behalf of
Amaury Levé (Evangelink)
September 1, 2026 10:33
View session
Contributor
🧪 Expert test review — PR #10895
All 18 new/modified test methods in this PR's This advisory comment was generated automatically. Grades are heuristic
|
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
Copilot review overview
Review tier: Balanced
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
|
src/Platform/Microsoft.Testing.Platform/Configurations/JsonConfigurationProvider.CommandLineOptions.cs — Defaults containing an array element of JSON null are still accepted even though the schema and… |
 |
src/Platform/SharedExtensionHelpers/ReportEngineBase.cs — The added TRX acceptance test does not exercise this shared implementation because… |
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
|
src/Platform/SharedExtensionHelpers/ReportEngineBase.cs — TryGetOptionArgumentListOrDefault returns true for a passive fallback too, so wasExplicit and… View resolved comment |
|
src/Platform/Microsoft.Testing.Platform.MSBuild/buildMultiTargeting/Microsoft.Testing.Platform.MSBuild.targets — When this item list transitions from non-empty to empty while a source testconfig.json still… View resolved comment |
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/Platform/Microsoft.Testing.Platform/ServerMode/JsonRpc/Json/Jsonite/JsonReader.cs:132
- This end-of-document check confuses the
Eofcharacter sentinel ('\0') with actual end-of-input. A document followed by a literal NUL byte leavesisEof == falsebutc == Eof, so the packaged MSBuild task accepts invalid trailing content that the System.Text.Json path rejects. Check the reader's EOF state instead.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f86614b5-b40f-415d-8f94-a3290fbe7154
Copilot started reviewing on behalf of
Amaury Levé (Evangelink)
September 1, 2026 11:12
View session
Contributor
🧪 Expert test review — PR #10895This PR adds passive
This advisory comment was generated automatically. Grades are heuristic
|
This comment has been minimized.
This comment has been minimized.
Contributor
There was a problem hiding this comment.
🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 147 AIC · ⌖ 2.89 AIC · ⊞ 16.9K · ◷
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f86614b5-b40f-415d-8f94-a3290fbe7154
Contributor
There was a problem hiding this comment.
Copilot review overview
Review tier: Balanced
Findings: None
Issues resolved since last review (2)
| Severity | Finding |
|---|---|
|
src/Platform/SharedExtensionHelpers/ReportEngineBase.cs — The added TRX acceptance test does not exercise this shared implementation because… View resolved comment |
|
src/Platform/Microsoft.Testing.Platform/Configurations/JsonConfigurationProvider.CommandLineOptions.cs — Defaults containing an array element of JSON null are still accepted even though the schema and… View resolved comment |
Suppressed comments (1)
src/Platform/Microsoft.Testing.Platform.MSBuild/Tasks/ConfigurationFileTask.cs:404
- The netstandard task only checks case-insensitive collisions among properties matching
requestedKey. A source such as{"platformOptions":{},"PlatformOptions":{}}, or duplicate-cased defaults unrelated to the MSBuild items being merged, therefore passes and is re-emitted even though the runtime configuration parser treats keys case-insensitively and rejects the generated file. Validate all property names in each visited object, matching theHashSetcheck in theNETCOREAPPbranch, before selecting the requested value.
foreach (KeyValuePair<string, object?> entry in jsonObject)
{
if (!string.Equals(entry.Key, requestedKey, StringComparison.OrdinalIgnoreCase))
{
continue;
Copilot started reviewing on behalf of
Amaury Levé (Evangelink)
September 1, 2026 11:25
View session
Contributor
🧪 Expert test review — PR #10895No new or modified test methods were identified in the changed regions Re-run with
|
Contributor
🧵 Parallel-safety audit — PR #10895Reviewed all 7 changed test files ( Every added/modified test:
Nothing to flag for parallel-safety. Audited:
Advisory only — heuristic, non-blocking. Re-run with
|
Amaury Levé (Evangelink)
deleted the
dev/amauryleve/testing-platform-option-defaults
branch
September 1, 2026 13:13
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
commandLineOptionDefaultssupport totestconfig.jsonTryGetOptionArgumentListOrDefaultso extensions can consume defaults without enabling their featureTestingPlatformCommandLineOptionDefaultMSBuild items that are projected into the built module configurationValidation
Fixes #6648