Modding, Mission Design, and Coding > FS2 Open Coding - The Source Code Project (SCP)

A suggestion for improved modding stability: Bugfix releases between stables.

<< < (3/4) > >>

mjn.mixael:

--- Quote from: Goober5000 on June 14, 2021, 10:20:42 am ---
--- Quote from: mjn.mixael on June 14, 2021, 08:31:16 am ---Not everything that causes compatibility issues is so clearly a bug. I've brought up this example before and at the risk of sounding like a broken record, I'm going to do so again because it's the perfect example of the kind of breakage mods encounter without expecting it. It's also a pretty good example of how SCP handles these types of issues (see: they don't).

BtA had a mission with an event tied to when the AI fires missiles. The idea was that the event would trigger when the AI hostiles get within firing range and actually start firing. There's a thousand ways to FRED such a thing, but this is the way the mission designer chose. This bug did not exist at release or for several years following. Then at some point unknown the AI started firing as soon as they arrive in the mission even though they are out of range of any valid target. This caused the event tied to AI missiles firing to trigger too early, breaking the mission.

It's easy to fix in FRED, which is what I did. But the implications of this small AI change are pretty broad. Outside of just the sexp, this implies that across the board the AI has fundamentally changed when they choose to fire missiles which can have far reaching effects on missions and mission balance. I know SCP is aware of the issue. Asteroth scolded me for months about not fixing the mission in BtA (Notice how he expected me to fix it in the mission even though it's the engine that broke it). Is anyone in SCP even interested in looking at why this behavior changed? I suspect not and that we're just gonna roll with it now.
--- End quote ---

If you're referring to the Mefistofele issue, I undertook an extensive debugging session, at the time that you posted the issue, to figure out the root cause.  It was caused by SEXPs, not AI, and it was a buggy SEXP that had its behavior masked by a bug in FSO.  (I believe Lepanto was the author of the buggy SEXPs in question.)

When the FSO bug was fixed, the SEXP bug revealed itself.  But I also described how the SEXP bug could be fixed, again at the time it was posted.  This is all documented on Discord.

--- End quote ---

Well I apologize then. Of course none of that Discord documentation is available to me. Thank you mageking.

Goober5000:
It's okay.  The discussion was more than a year ago so one wouldn't necessarily remember the details.

Neither of the issues you mentioned above are within my usual areas of expertise, but I might be able to figure out 3367, so I'll take a look at that.

jg18:

--- Quote from: Goober5000 on June 14, 2021, 10:20:42 am ---
--- Quote from: Phantom Hoover on June 14, 2021, 09:39:29 am ---
--- Quote from: Goober5000 on June 13, 2021, 07:11:39 pm ---It doesn't take much effort to post an official build, but it does take a fair amount of effort to manage the release cycle.  I don't know that we have enough manpower to produce maintenance releases with everything else going on.  PR reviews take a lot of work too, and there are a lot of pull requests currently outstanding that need reviewing.

--- End quote ---

The proposal here is basically to make minor patch releases to stable builds if and when regressions are caught.
--- End quote ---

I don't disagree with the proposal, per se, I'm just saying that it would involve a certain amount of work.  In addition to branching and patching, one would have to collect a list of bugs, filter out the fixes to be applied, and merge them.

--- End quote ---
While I'm glad Goober agrees on the merits of the idea, I do have to agree with him that it would take a non-trivial amount of dedicated work to keep a series of maintenance releases going, and finding someone among the current active SCP coders who can do that is hard. :blah: Unless, of course, someone currently not on SCP would like to step up? ;7 I don't think it'd require any serious/deep coding skills, just ability/willingness to learn and dedicate to it. Just saying :nod:

Mjn, I don't know anything about the code pertaining to Issue #3375, but I can take a look. Maybe I can at least narrow down the cause.

z64555:
  Well I had the fun of going through the forums and the two SCP channels on discord to get up to speed on this discussion, but I believe I have enough information now to grasp what is going on.

  The proposal here is to add an official build that is based on a stable release and includes bugfixes to critical game-breaking issues that mods may stumble upon between release cycles, thus called a point release (<year>.<quarter>.<point>).  This is to address unexpected issues that show up and are critical to the health of a mod, especially regressions that break older, already released mods.

So lets lay down some points:

* Point releases are bugfix-only builds
* Point releases are done outside the normal release cycle process, but are treated in similar modus operande
* Point releases are done only on the most recent stable release
* Point releases are done by the discretion of the team
* Point releases have the option of reverting a feature entirely for the sake of stability

Item 1:  Point releases are bugfix only builds.
  No additional features, no extra QOL.  If the client wants either new feature or new QOL feature, then they'll have to wait until the next release.  This is because we want the point release to isolate off any possible vector of a new bug from occurring, which is largely from new features but also from previous bugfixes.  We can't predict if a bugfix nor a feature is the cause of a new bug, but isolating fixes from features should help keep the team focused on the not-fun gruntwork that bugfixing is.


Item 2: Point releases are done outside the normal release cycle process, but are treated similarly.
  To keep things simple, the "point release cycle" should work in a very similar fashion to what we do with the release cycle.  That is:
1.  Make the fix
2.  Publish the fix as a candidate
3.  Test it for a week
4.  If no further bugs found, and the fix is confirmed as working, the candidate is released and the point version is incremented by 1.
5.  If new bugs were found or the fix did not work fully, repeat 1 through 5.

Now, some key similarities and differences here:

* Point release cycles are all-hands-on-deck situations, as are Release cycles.
* Point releases are essentially feature frozen just as with release cycles, but master is not feature frozen.
*
* This implies that point releases have their own branch based on the published released that diverge from master.  Master can still have features worked on and merged into it while a point release is going on, but its not recommended to do so (since everybody should be focused on the bugs).
* Point releases are stable, and are only based on a stable release build or another point release
* Point release cycles are "its done when its done," just like with release cycles.  The difference here is that only critical bugs and fixes will be addressed.
*
* Definition of what constitutes a critical bug is explained later
* Release cycles are planned, scheduled events.  Point release cycles are planned, unscheduled events and are irregular in nature.

Item 3:  Point fixes are done only on the most recent stable.
  Bugfixing is not fun for coders.  It really isn't.  It can be hard, grueling, and frustrating to track down what's causing a bug, and sometimes it is even an adventure to reproduce the bug reliably in the first place.  I'm not inclined to backport bugfixes from a new stable release into an older stable because somebody doesn't want the features of a newer build.  For that matter, if a new feature is neither opt-in nor opt-out, it probably should not have been merged in the first place.

  Item 3 also has an effect on the repository depending on the branching strategy we wind up with.  If we go with having a special release branch that's based on the stable release but is isolated from master, then we can wind up with a number of vestigial branches that are left behind (as compared to having a simple, lightweight collection of tags).  Merging the fixes from the point release branch into master doesn't make the point release branch go away.

Note: we still should attempt merging the fixes from the point release back into master so the fix doesn't have to be remade.  This is not guaranteed to always work without conflicts, however.

  If we go with having a dev branch (master) and a release branch that runs parallel to master but maintained,  we can use tags on the release branch.  Essentially, the release branch is all stable and after a release cycle is finished, the features are logically squashed and tidied and then pushed from dev into release.  Should a point release be needed, the fixes are branched off and merged back into the release branch directly.

  The dev/release dual trunklines is nice and tidy in theory, but it requires a higher degree of management than does the single dev branch and tagged releases, and risks having unstable commits accidentally merged into stable without careful review of where the commit is going. (Github doesn't make it clear enough that "User wants to merge into Master" and "User wants to merge into Stable" with their UI).  Additionally our automation may need changes to support the switch.


Item 4.  Point releases are done by the discretion of the SCP team.
  If the team feels that it's better to wait for the next release cycle to start than to try to start a point-release cycle, then there's no need to start a point release cycle.  This is a grey area depending on the state of its team members (yes, we are a volunteer organization.  No, an emergency for you is not strictly an emergency for me), and exhaustive back-to-back point release cycles will make the team less likely to put their all into the next stable release cycle.  Release cycles will always be our priorities, since they are more regular and thus can be scheduled for time-off to focus from paying jobs and personal schedules should the team members decide to do so.  Point release cycles are only for special circumstances and should be kept at a minimum, since some team members that have been scheduled to be elsewhere won't be able to contribute to the point release cycle, and at worse cause a deadlock because of specialist knowledge about the affected code.


Item 5.  Point releases have the option of reverting a feature entirely for the sake of stability
  If a long point release cycle occurs, the team may decide to revert a feature that is causing them.  This feature will then have to wait for the next release cycle to be available on a stable build, thus giving time for its creator to rethink its design.  This is to try to make a stable build, actually stable.  If a feature is so bug ridden or otherwise a pain to work with, then the team shouldn't have to exert excessively.  As with Item 4, this option is at the discretion of the team.


Now, for managing the point release cycle.

* A new priority column in the Bug Triage kanban will be made for critical bugs, the kanban process for bug triage is otherwise the same. (Diagnose, Prioritize, Test a fix, Publish)
* "Point Release Candidate" and "Not a Point Release Candidate" tags go away.  These are not helpful in the issue tracker after they've been merged in and can increase tag clutter.  Especially since priorities are already given in the Bug Triage kanban
* Point release cycles are special, irregular circumstances that are entered the moment a critical bug is identified and does not have a feasible workaround that'll keep a mod going until the next release cycle.  The Support Ninjas will be deployed and assassinate the critical bug ASAP.
* SCP members on discord will get a ping when a critical bug is diagnosed and confirmed in the kanban.  This is different from regular release cycles because critical bugs are special circumstances and need to be fixed yesterday.


Finally, what defines a critical bug?
  A critical bug is one in which breaks expected behavior of any mod that has been published and released (thus, a regression), or
a bug that significantly blocks the development of a new mod and can not have a feasible workaround to keep it working until the next release cycle.

  Regressions are especially nasty to the SCP if ignored, as it demoralizes players and its creators and in general makes us look like assholes.  Dev-stopping bugs for new mods are similarly bad for our public relations, as its creators can have enough stress as-is trying to make the mod they want without needing the additional stress from an indifferent, uncaring monolith they have to plead with to get their mod back on track.  This, along with a "stable that isn't stable" goes against the moral of the community and can make the place not fun to be in if the behavior becomes expected, leading to community death.

  While this may seem like an aside, the goal of point-releases is to reduce the stress on players and mod makers so they can get back to playing the game they love so much without needing to cuss out or plead the SCP to fix their beloved again.  And, for that matter, to better retain players and mod makers in a community that "actually cares."

EatThePath:

--- Quote from: z64555 on June 17, 2021, 11:08:23 pm ---  Regressions are especially nasty to the SCP if ignored, as it demoralizes players and its creators and in general makes us look like assholes.  Dev-stopping bugs for new mods are similarly bad for our public relations, as its creators can have enough stress as-is trying to make the mod they want without needing the additional stress from an indifferent, uncaring monolith they have to plead with to get their mod back on track.  This, along with a "stable that isn't stable" goes against the moral of the community and can make the place not fun to be in if the behavior becomes expected, leading to community death.

  While this may seem like an aside, the goal of point-releases is to reduce the stress on players and mod makers so they can get back to playing the game they love so much without needing to cuss out or plead the SCP to fix their beloved again.  And, for that matter, to better retain players and mod makers in a community that "actually cares."

--- End quote ---

I'm not going to have the ability to usefully respond to all of this till the weekend at least, but I just want to say this hits the nail on the head on why I proposed this.  I think everyone cares here, in their own ways, and I hope efforts like this can help solidify that.

Navigation

[0] Message Index

[#] Next page

[*] Previous page

Go to full version