Skip to content

Fix opendata-ci auto-commit gap and add ISF/eligibility PowerShell commands - #2298

Open
Michael Flanakin (flanakin) wants to merge 5 commits into
devfrom
flanakin/opendata-ci-untracked-files
Open

Fix opendata-ci auto-commit gap and add ISF/eligibility PowerShell commands#2298
Michael Flanakin (flanakin) wants to merge 5 commits into
devfrom
flanakin/opendata-ci-untracked-files

Conversation

@flanakin

Copy link
Copy Markdown
Collaborator

🛠️ Description

opendata-ci.yml used git commit -a after regenerating PowerShell functions from src/open-data/*.csv. git commit -a only stages modified/deleted tracked files — it silently skips new untracked files. InstanceSizeFlexibility.csv (#2199) and CommitmentDiscountEligibility.csv (#2251) never had their generated Get-OpenData*.ps1 functions committed, so they stayed untracked forever and every PR touching either CSV (most recently #2295) saw a "no diff" CI result that masked the standing gap. Fixed by switching to git add -A before git commit.

Also adds Get-FinOpsInstanceSizeFlexibility and Get-FinOpsCommitmentDiscountEligibility public cmdlets, matching the existing Get-FinOpsPricingUnit/Get-FinOpsService pattern — these two datasets had no PowerShell-facing lookup at all, only direct CSV consumption (Power BI, Optimization Engine).

Bugs found and fixed along the way

  • Get-FinOpsCommitmentDiscountEligibility's initial draft used the same Select-Object -Property * -Unique post-filter as the other Get-FinOps* cmdlets. At 92k+ rows this hung for minutes. MeterId is already unique in the source data, so -Unique was dropped (both from the cmdlet and its test's BeforeAll).
  • Build-OpenData.ps1's Write-Command/Out-File generation path silently dropped rows on large datasets on some runs (662 rows missing from one CommitmentDiscountEligibility generation, 1 row from an InstanceSizeFlexibility generation). Regenerating fixed it both times. Not root-caused further — flagging here as a latent reliability risk in the generator for anyone touching it next.
  • Merging in dev picked up fix(open-data,hubs): align commitment discount eligibility with FOCUS spend and usage categories #2284, which swapped which eligibility column maps to reservations vs. savings plans. Corrected the new cmdlet's help text/docs to match and regenerated the private function against the updated CSV.

Fixes # (opendata-ci not running as expected on PRs like #2295)

📷 Screenshots

N/A — PowerShell module change

📋 Checklist

🔬 How did you test this change?

  • 🤏 Lint tests
  • 🤞 PS -WhatIf / az validate
  • 👍 Manually deployed + verified
  • 💪 Unit tests
  • 🙌 Integration tests

📦 Deploy to test?

  • Hubs + ADX (managed)
  • Hubs + Fabric (manual) — URI:
  • Hubs (manual)
  • Hubs (no data)
  • Workbooks
  • Alerts

🙋‍♀️ Do any of the following that apply?

  • 🚨 This is a breaking change.
  • 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

  • ✅ Updated changelog (required for dev PRs)
  • ➡️ Will add log in a future PR (feature branch PRs only)
  • ❎ Log not needed (small/internal change)

📖 Did you update documentation?

  • ✅ Public docs in docs (required for dev)
  • ✅ Public docs in docs-mslearn (required for dev)
  • ✅ Internal dev docs in docs-wiki (required for dev)
  • ✅ Internal dev docs in src (required for dev)
  • ➡️ Will add docs in a future PR (feature branch PRs only)
  • ❎ Docs not needed (small/internal change)

🤖 [AI] Generated with Claude Code

…commands and fix opendata-ci auto-commit gap

Build-OpenData.ps1 -PowerShell already generates Get-OpenData* functions for
every CSV in src/open-data/, but InstanceSizeFlexibility.csv and
CommitmentDiscountEligibility.csv (added in #2199/#2251) never had their
generated Private/ functions committed, and neither had a public Get-FinOps*
cmdlet like the other open data (PricingUnit, Region, ResourceType, Service).

Root cause: the opendata-ci.yml workflow used `git commit -a`, which only
stages modified/deleted tracked files and silently ignores new (untracked)
files. Since these two datasets' generated .ps1 files had never been
committed, they stayed untracked forever and `git commit -a` never picked
them up, so CI's "no diff" result masked a standing gap on every PR that
touched either CSV (most recently #2295). Switched to `git add -A` before
committing.

Adds Get-FinOpsInstanceSizeFlexibility and
Get-FinOpsCommitmentDiscountEligibility public cmdlets following the existing
Get-FinOpsPricingUnit/Get-FinOpsService pattern, plus docs and TOC entries.

Get-FinOpsCommitmentDiscountEligibility drops the `-Unique` post-filter used
by other Get-FinOps* cmdlets: MeterId is already unique in the source data
(92k+ rows), and Select-Object -Unique over that many PSCustomObjects was
measured to hang for minutes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…merged CSV

#2284 corrected which CommitmentDiscountEligibility column maps to
reservations vs. savings plans (x_CommitmentDiscountSpendEligibility =
savings plan/spend commitment, x_CommitmentDiscountUsageEligibility =
reservation/usage commitment). Fixed the same mix-up in the new
Get-FinOpsCommitmentDiscountEligibility cmdlet's help text and docs page.

Also regenerated Get-OpenDataCommitmentDiscountEligibility.ps1 against the
CSV that came in through the #2284 merge (72k rows changed) to keep the
committed private function in sync with the source data.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@flanakin
Michael Flanakin (flanakin) marked this pull request as ready for review September 2, 2026 15:55
Copilot AI lite review requested due to automatic review settings September 2, 2026 15:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A few newly added/edited docs strings are internally inconsistent with the cmdlet defaults and contain an incorrect command description that should be corrected before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR fixes the Open Data CI auto-commit behavior so newly generated PowerShell files are actually staged/committed, and adds new PowerShell cmdlets + docs/tests to expose the Instance Size Flexibility and Commitment Discount Eligibility datasets via the FinOpsToolkit module.

Changes:

  • Update opendata-ci.yml to git add -A before committing generated files so untracked outputs are not silently skipped.
  • Add Get-FinOpsInstanceSizeFlexibility and Get-FinOpsCommitmentDiscountEligibility public cmdlets (and corresponding unit tests).
  • Add/refresh generated Get-OpenData* private functions + unit tests and wire up Microsoft Learn docs/TOC entries.
File summaries
File Description
.github/workflows/opendata-ci.yml Fixes CI auto-commit to include newly generated (previously untracked) files.
src/powershell/Public/Get-FinOpsInstanceSizeFlexibility.ps1 New public cmdlet for querying ISF open data with wildcard filters.
src/powershell/Public/Get-FinOpsCommitmentDiscountEligibility.ps1 New public cmdlet for querying commitment discount eligibility by meter ID and eligibility flags.
src/powershell/Private/Get-OpenDataInstanceSizeFlexibility.ps1 Generated private lookup function for ISF dataset.
src/powershell/Private/Get-OpenDataCommitmentDiscountEligibility.ps1 Generated private lookup function for commitment discount eligibility dataset.
src/powershell/Tests/Unit/Get-OpenDataInstanceSizeFlexibility.Tests.ps1 Unit test ensuring generated ISF function matches CSV row count.
src/powershell/Tests/Unit/Get-OpenDataCommitmentDiscountEligibility.Tests.ps1 Unit test ensuring generated eligibility function matches CSV row count.
src/powershell/Tests/Unit/Get-FinOpsInstanceSizeFlexibility.Tests.ps1 Unit tests for new public ISF cmdlet behavior and typing.
src/powershell/Tests/Unit/Get-FinOpsCommitmentDiscountEligibility.Tests.ps1 Unit tests for new public eligibility cmdlet behavior and filters.
docs-mslearn/toolkit/powershell/powershell-commands.md Adds the new cmdlets to the PowerShell command reference list.
docs-mslearn/toolkit/powershell/data/open-data-commands.md Adds the new cmdlets to the open data command list.
docs-mslearn/toolkit/powershell/data/get-finopsinstancesizeflexibility.md New Microsoft Learn reference page for the ISF cmdlet.
docs-mslearn/toolkit/powershell/data/get-finopscommitmentdiscounteligibility.md New Microsoft Learn reference page for the eligibility cmdlet.
docs-mslearn/toolkit/open-data.md Adds “See PowerShell” links for the two datasets.
docs-mslearn/TOC.yml Adds TOC entries for the new cmdlet docs pages.
Review details

Suppressed comments (1)

src/powershell/Public/Get-FinOpsCommitmentDiscountEligibility.ps1:16

  • The comment-based help for UsageEligibility says the default is null, but the parameter default is "*" and the filter uses -like. Update the help text so it matches the actual default behavior.
    .PARAMETER UsageEligibility
    Optional. Filters to meters with the specified reservation (usage commitment) eligibility. FOCUS classifies a reservation as a usage commitment because you commit to a quantity of usage. Expected values: Eligible, Not Eligible. Default = null (all).

  • Files reviewed: 13/15 changed files
  • Comments generated: 4
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +36 to +37
| SpendEligibility | Optional. Filters to meters with the specified savings plan (spend commitment) eligibility. FOCUS classifies a savings plan as a spend commitment because you commit to an amount of money. Expected values: Eligible, Not Eligible. Default = null (all). |
| UsageEligibility | Optional. Filters to meters with the specified reservation (usage commitment) eligibility. FOCUS classifies a reservation as a usage commitment because you commit to a quantity of usage. Expected values: Eligible, Not Eligible. Default = null (all). |

- [Get-FinOpsCommitmentDiscountEligibility](Get-FinOpsCommitmentDiscountEligibility.md) – Gets commitment discount eligibility for a meter.
- [Get-FinOpsInstanceSizeFlexibility](Get-FinOpsInstanceSizeFlexibility.md) – Gets the instance size flexibility group and ratio for an ARM SKU.
- [Get-FinOpsPricingUnit](Get-FinOpsPricingUnit.md) – Gets an Azure region ID and name.

- [Get-FinOpsCommitmentDiscountEligibility](data/Get-FinOpsCommitmentDiscountEligibility.md) – Gets commitment discount eligibility for a meter.
- [Get-FinOpsInstanceSizeFlexibility](data/Get-FinOpsInstanceSizeFlexibility.md) – Gets the instance size flexibility group and ratio for an ARM SKU.
- [Get-FinOpsPricingUnit](data/Get-FinOpsPricingUnit.md) – Gets an Azure region ID and name.
Comment on lines +11 to +13
.PARAMETER SpendEligibility
Optional. Filters to meters with the specified savings plan (spend commitment) eligibility. FOCUS classifies a savings plan as a spend commitment because you commit to an amount of money. Expected values: Eligible, Not Eligible. Default = null (all).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Review 👀 PR that is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants