Proposal for having a single feature band going forward - #373
Proposal for having a single feature band going forward #373marcpopMSFT wants to merge 6 commits into
Conversation
…l configuration that all tools rely on to get new feature work.
| - A feature starts as preview-on, stable-off, baseline-off. | ||
| - Promotion to stable requires explicit approval and release notes in a planned update. | ||
| - Baseline remains off by default, with opt-in available for customers that explicitly choose it. | ||
| - Minimum preview soak time before promotion is two months. |
There was a problem hiding this comment.
Two months seems a bit long for low-risk changes. UPDATE: I see this was mentioned as well in the Risks and Unknown section
There was a problem hiding this comment.
I defaulted to two months as that's roughly the timeline VS has for a change going from canary to stable. I think it's up to teams to decide how long they need depending on the feature as some features they may want longer and some things could be shorter. <1 month is probably not possible unless we change our release schedule.
There was a problem hiding this comment.
Ah ok I see, maybe 1 month minimum is the sweet spot
| 3. Host-computed default written by SDK startup | ||
| 4. SDK build-time default | ||
|
|
||
| Global.json integration may be added later, but is not required for the initial design. |
There was a problem hiding this comment.
| Global.json integration may be added later, but is not required for the initial design. | |
| global.json integration may be added later, but is not required for the initial design. |
I understand why you maybe don't want to add this initially but this was the method I was hoping to use so I could share the config with the other developers and know that CI will use the same config.
There was a problem hiding this comment.
We'll discuss with our team on what the design looks like for how customers enable this. I'm most concerned about ensuring tools don't accidentally make breaks in stable or baseline with how to enable this setting being second to that. For most customers though, I would hope this would be a noop where they wouldn't have to think about it and wouldn't want to change.
There was a problem hiding this comment.
I look forward to hear what you come up with. I don't understand how it would affect accidentally making breaks could you elaborate? So you are hoping the default for most teams would be baseline right? Will baseline be the default?
There was a problem hiding this comment.
To clarify, if we switch to a model with a single band, I have to ensure that all partner teams that contribute to the SDK follow this model. We've had a hard time in the past ensuring everyone followed the breaking change model for feature bands and this makes the consequences worse of having a breaking change if it slips into baseline.
How customers configure this is the second item on my list to figure out. This is a very early proposal but I wanted to get it out to get feedback before we got too far along.
|
|
||
| Unknowns to validate during review: | ||
|
|
||
| - Whether `baseline` is intuitive enough for external customers and partners. |
There was a problem hiding this comment.
Baseline is the most stable so I guess there could be some confusion, but I worry that some will misunderstand the guarantees given by baseline and stable and unnecessarily choose(mandate) baseline (irrespective of the name but naming helps alot with conveying intent). So by itself baseline is understandable but also having a channel named stable makes it maybe less obvious.
There was a problem hiding this comment.
I had to think again when responding to some comment about what baseline and stable represented so maybe an indication that it will take some time to get used too or that other words would be better but I haven't come up with any yet.
There was a problem hiding this comment.
Thanks, we'll have to keep brainstorming here to see if we can come up with something that doesn't require a lot of thinking. I picked what I picked initially as we already have previews and VS monthly releases are called the stable channel already. baseline was a best effort.
|
@zivkan as we've touched on feature roll out/behavior changes for some of our discussions. |
|
|
||
| **Owner** [marcpopMSFT](https://github.com/marcpopMSFT) | ||
|
|
||
| Today, SDK feature bands, SDK tools, and Visual Studio are coupled in ways that is complex to manage and confusing for users. The challenge with changing this is users can receive new features or behavior changes without intentionally opting in. This is especially problematic for managed environments and source build partners that prioritize stability. We propose a single SDK feature band strategy, combined with a global feature channel setting that all SDK-loaded components honor. This setting has three states: baseline, stable, and preview. New behavior is always disabled in baseline, and initially enabled only in preview. |
There was a problem hiding this comment.
I don't quite follow how introducing a new concept (channel setting) makes it less complex.
I'd be good to explain why staying on older SDK version that only receives security fixes isn't simpler and sufficient for customers who need stability. Is it because VS is tied to the latest SDK?
There was a problem hiding this comment.
You're right that there is risk of us making this more complicated where there's a 1xx version that could have 3 different behaviors though we already have plenty of features you can opt into and out of. I think the confusion is customers not knowing which band they have or what the bands mean. It also simplifies some of our internal engineering though does add some separate branch complexity.
The main simplification is there's just be a 1xx build and so customers could just install that and not actually have to think about it. Most customers would never change the actual setting for this as source build partners would default to baseline, stable VS customers would default to stable, VSCode customers would default to stable, and insiders/canary customers would be preview. Most customers would probably just stick with the default.
|
cc @dotnet/distro-maintainers |
| Open questions: | ||
|
|
||
| - Do we need an enfrocement bot enabled in every participating repo? | ||
| - Do we need the ability to dump all new features into a machine and human readable format for review and comparison against intent/docs/PRs? |
There was a problem hiding this comment.
Related, we should do a good job at documenting in release notes what channel new features are included in. That includes when a release includes promotion of an existing feature to a different channel.
There was a problem hiding this comment.
Slightly related if this would be available for Issues and PRs on GitHub as well that would help with planning for when we are dependant on a fix or feature being made available so maybe some new tags as I guess the branch would only be N.0.1xx or similar
There was a problem hiding this comment.
We'll have to brainstorm on how to track this and doc this. it's easy for an ai to look at a build release and see which features are in baseline, stable, and preview. Tying that back to specific PRs/issues is probably still pretty feasible for a computer to do. We'll have to explore tooling as we don't want the management of this to become a toil.
|
|
||
| 1. Whether `DOTNET_SDK_FEATURE_CHANNEL` should be the only v1 control surface or whether a CLI switch should also ship in v1. | ||
| 2. Minimum schema for the machine-readable behavior manifest. | ||
| 3. Is there a change to the versioning schema we could do for the SDK? We could change to match runtime versioning but we sometimes have SDK hotfixes so it might be confusing to release an 11.0.108 that has 11.0.7 runtime in it. No newline at end of file |
There was a problem hiding this comment.
In my experience the SDK versioning has been a source of confusion. Eliminating the multiple feature bands will help significantly. What is the need to retain 1xx vs xx? Is the thought that if SDK needs a hotfix having misalignment between the SDK and runtime will cause confusion? e.g. runtime 11.0.7 vs sdk 11.0.8 will be confusing but runtime 11.0.7 vs sdk 11.0.108 avoids this?
There was a problem hiding this comment.
I raised something similar here https://github.com/dotnet/designs/pull/373/changes#r3688674762 but yes I guess you brought up the main concern with the hotfixing
There was a problem hiding this comment.
Correct, hotfixing is the concern and them getting out of sync. I don't know if that would cause more confusion than having the 1 on front and then having them be out of sync. (ie is .108 having the .7 runtime less confusing than .8 with the .7 runtime)?
There was a problem hiding this comment.
Thinking more on this, there's a lot of existing code that assumes feature bands like workloads code and global.json parsing. I think it may be more trouble than it's worth to get rid of the band versioning.
There was a problem hiding this comment.
I think for me when I help coworkers it will be easier for me to detect when they give me the runtime version instead of SDK version that I asked for if this follows the current versioning.
There was a problem hiding this comment.
@MichaelSimons an alternative proposal added to the doc would be to switch to minor releases. The drawback is that we'd lose the benefit this proposal has of enabling customers who use the source build packages to opt into newer features which is unfortunate. It does let us simplify the versioning somewhat but I don't know if minors are more intuitive (they'd be monthly and probably we'd stop doing minor after a year and switch to patches once there were no more feature changes coming in).
Thoughts on this tradeoff?
|
@marcpopMSFT Can you add a scenario for Microsoft Update? Currently feature bands are serviced separately. Someone on 1xx will only be offer 1xx updates. But if they are on 2xx or 4xx, they'll get those too. SDKs from VS are serviced as from VS. The standalone SDK includes all the runtimes so those fixes are rolled up and a single offering can target multiple products. |
| @@ -0,0 +1,251 @@ | |||
| # Single SDK Feature Band with Global Feature Channels | |||
There was a problem hiding this comment.
If there's only ever a single feature band, can't just the entire "feature band" concept go away?
There was a problem hiding this comment.
As covered above, there are SDK hotfixes that have no runtime release. Is a .108 SDK containing a .7 runtime more or less confusing than a .8 SDK with a .7 runtime? We typically do 1-3 SDK only hotfixes in a year and probably only the first year as our focused shifted to VNext.
There was a problem hiding this comment.
Can we keep the 1xx versioning but stop talking about it as "feature band" concept? (Since we have only one "feature band", we don't need it as a concept, it seems to be making things unnecessarily complicated.)
There was a problem hiding this comment.
Yes, that was exactly my point, we can keep versioning the same way, just say it's "how we version" and not use a "feature band" concept.
There was a problem hiding this comment.
I think this is a nice middle ground that could work. We wouldn't have to figure out how to change our whole infrastructure which is what we'd need if we did a 11.0.0 or a 11.1.0 versioning of the SDK and we'd avoid the mismatched issue with runtime. We just call it the SDK version. global.json would still have rollforward feature but that wouldn't really apply going forward anymore.
@joeloff can you provide more info on what you're looking for? Us collapsing into a single band means we'd only offer MU for that one band. |
| - PR review guidance as a minimum | ||
| - What other enforcement can we create? | ||
| - Do we need the ability to dump all new features into a machine and human readable format for review and comparison against intent/docs/PRs? | ||
| - It'd be great to be able to tag issues/PRs on which release they are in as well. |
There was a problem hiding this comment.
release here meaning baseline, stable or preview? When I hear release I think about as an example 10.0.302 or 10.0.10. GitHub natively currently show which GitHub release an issue is fixed in but this isn't extended to PRs yet but hoping that will come sooner rather than later so the largest benefit here would be to know ahead of time which release something will land in and what configuration baseline, stable or preview.
There was a problem hiding this comment.
features would never land in baseline presumably. But the idea would be to tie a new feature/setting to the patch version that it showed up in as stable. I think that's the most important piece to track as that's when most customers would end up getting the feature. Tracking when it released in preview is probably next most interesting. .NET has milestones we use for servicing and that's what I meant when extending back to PRs and issues.
…shipping in the SDK be stable versions.
with a central configuration that all tools rely on to get new feature work.