Author Topic: Persistent variables  (Read 7180 times)

0 Members and 1 Guest are viewing this topic.

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Persistent variables
As a result of another thread, I became aware that player-persistent variables have been broken since Antipodes 8 (the pilot code overhaul that was the major new feature in FSO 3.7.0). Namely, they're actually player-persistent now, instead of resetting when the campaign does.

"Wait, what?" you might ask. "Player-persistent variables being player-persistent is a bug?"

Yes. Yes it is.

When player-persistent variables were first introduced (all the way back in 2003), it wasn't really possible to play more than one campaign simultaneously. Whenever you changed campaigns, PPVs would be explicitly reset. This reset was removed at the start of Antipodes 8 back in 2010, but without any attempt made to keep the old behavior in some way. Presumably simply never resetting PPVs was considered preferable to them resetting if you just switched to a different campaign and then switched back, but the proper solution would have been to move them to the .csg file instead of leaving them in their current state.

Alas, they were left in their current state, and have been so for years (3.7.0 Final was released in August of 2013).

No documentation appears to have been updated to account for the code change; the wiki still claims player-persistent variables get reset with the campaign. Why haven't I corrected the wiki? Well, because the way the wiki describes it is definitely how it's supposed to work, and how FREDers have been using them since they were introduced. The difference that's supposed to be between the two variable types is when they're saved: CPVs are saved at the end of the mission (assuming you accept the outcome), while PPVs are updated as they change (so restarting a mission after it sets a PPV will leave the PPV set to the new value).

Now, I'm currently thinking that the correct fix here is to move so-called "player-persistent variables" into the campaign savefile, even though that would require bumping the pilotfile version to accommodate this change, because as they're intended to be used, they're definitely still supposed to be tied to an individual campaign.

On the other hand, I'm sure designers would like variables that could be accessed between campaigns, so at least one new type of variable needs to be added with that functionality (the current, actually-wrong behavior of player-persistent variables). However, renaming PPVs and then making a new type of variable and calling them PPVs is just asking for somebody to get confused and pick the wrong one. So ideally, we need all new names for every kind of persistent variable so that there's no confusion about what they actually do and what they should actually be used for.

Karajorma suggested that 4 types of variables are needed, with the two axes being campaign-/player-specific and saved-on-accept/saved-on-change. Thinking about it as I write this post, perhaps there might even be a way to incorporate wookieejedi's request for a kind of variable that is persistent even through techroom runs; 8 kinds of variables?

Regardless, here's my initial suggestion for the new naming scheme:
  • "Campaign-persistent variables" would become "Progression variables" (tied to the campaign, saved when accepting the mission outcome).
  • "Player-persistent variables" (the way they're supposed to work) would become "Persistent variables" (not fond of this name, but it was the first thing that came to mind; if you have a better idea, please suggest it) (tied to the campaign, saved whenever the value changes)
  • "'True' player-persistent variables" (the incorrect way they currently work) would be "Eternal variables" (Karajorma's idea, but it makes sense to me) (tied to the player, saved whenever the value changes)
  • No idea what to call player-specific variables saved on accepting mission outcome; please suggest something.

If "techroom-usable" variables were a separate type (rather than a flag that could be set for one of the other kinds of variables, for example), I have no idea what they'd be called.

Thoughts? Comments? Completely-off-the-wall ideas? Anyone? Bueller?
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline karajorma

  • King Louie - Jungle VIP
  • Administrator
  • 214
    • Karajorma's Freespace FAQ
Re: Persistent variables
Given that the only difference between PPVs and CPVs as designed is whether or not they reset on a failed mission it strikes me that a better option is to leave things at only two types of persistent variables but then have a flag decide whether to write it to the player or campaign file. An advantage of this approach is that if someone thinks of another type of persistent variable we wouldn't need to come up with two new names for the player and campaign file versions.

Another major advantage of that would be that it would safely define a system for removing variables from the player file. If you had something like this

Mission 1 - Variable is Progressive and Eternal
Mission 2 - Variable is Progressive and Eternal
Mission 3 - Variable is Progressive

In mission 1 and 2 the value of the variable is written to the player file when the mission is successfully completed. But in Mission 3 the game detects that the Eternal flag has been switched off so the value would be written to the campaign file and the variable in the player file would be deleted on mission completion.
Karajorma's Freespace FAQ. It's almost like asking me yourself.

[ Diaspora ] - [ Seeds Of Rebellion ] - [ Mind Games ]

 

Offline 0rph3u5

  • 211
  • Oceans rise. Empires fall.
Re: Persistent variables
Question:

Can this be paired with a convinence function to import variables form one mission to another in FRED?
"As you sought to steal a kingdom for yourself, so must you do again, a thousand times over. For a theft, a true theft, must be practiced to be earned." - The terms of Nyrissa's curse, Pathfinder: Kingmaker

==================

"I am Curiosity, and I've always wondered what would become of you, here at the end of the world." - The Guide/The Curious Other, Othercide

"When you work with water, you have to know and respect it. When you labour to subdue it, you have to understand that one day it may rise up and turn all your labours into nothing. For what is water, which seeks to make all things level, which has no taste or colour of its own, but a liquid form of Nothing?" - Graham Swift, Waterland

"...because they are not Dragons."

 
Re: Persistent variables
Going off karajorma's suggestion here... I think we could use a system of flags like this:

Persistent: Must be set for the variable to be persisted at all; without this, only the shadow flag has meaning.
Persist regardless of outcome: Writes the variable even if the mission was a failure.
Visible between campaigns: Whether to write the variable to the pilot file (true) or campaign file (false).
Visible in simulator:  Whether the tech room simulator can see the value of this variable.
Persist from simulator: Whether the simulator can modify this variable. Must be set along with the previous flag.
Allow shadowing: If not set, the game issues a warning if this variable has the same name, but different flags or type, as another variable in the same campaign.

All flags are false by default.
« Last Edit: May 13, 2017, 07:12:48 am by xenocartographer »

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: Persistent variables
Allow shadowing: If not set, the game issues a warning if this variable has the same name, but different flags or type, as another variable in the same campaign.

This makes little sense to me. Allowing variables to be shadowed leads to complications that are best avoided entirely.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 
Re: Persistent variables
Wait... so if this changes that means you won't have to manually delete savefiles when switching to a different version of a campaign with the same name? Because right now if you play campaign A beta v.091 and then switch to campaign A beta v.096 your pilot save file becomes invalid and you either need to delete it or create a new pilot.


Player-specific variables saved on accepting mission outcome are kills and ranks, right? Why not just call them statistics variables or achievement variables?
[19:31] <MatthTheGeek> you all high up on your mointain looking down at everyone who doesn't beam everything on insane blindfolded

 
Re: Persistent variables
Allow shadowing: If not set, the game issues a warning if this variable has the same name, but different flags or type, as another variable in the same campaign.

This makes little sense to me. Allowing variables to be shadowed leads to complications that are best avoided entirely.

I can imagine cases where someone would want different values for the "visible in simulator" or "persist regardless of outcome" flags.

FrikgFeek, we're talking about SEXP variables, not killcounts - stuff like your treatment of the Gefs in Nothing is True (though that's probably done through is-previous-event-true, but whatever).

 
Re: Persistent variables
Wouldn't that be the first case? Campaign persistent? I guess I just don't understand what the 4th case would be then. Saved on accepting mission outcome but not campaign persistent? What's player-specific even supposed to mean compared to player-persistent?
Is it possible to have campaign persistent variables that aren't player-specific?
[19:31] <MatthTheGeek> you all high up on your mointain looking down at everyone who doesn't beam everything on insane blindfolded

 
Re: Persistent variables
I'm... I'm confused. Could you try and give me some context for your question? I'm not sure we're talking about the same things here.

 
Re: Persistent variables
Quote
"No idea what to call player-specific variables saved on accepting mission outcome; please suggest something."


How is this 4th case different from the 1st case?
Quote
"Campaign-persistent variables" would become "Progression variables" (tied to the campaign, saved when accepting the mission outcome).

They're both saved on mission outcome and presumably campaign specific? Unless the 4th case is 'eternal' but saved on mission outcome? I'm not even sure what kinds of SEXP variables would be used like that. That's what I'm confused about. That's why I (wrongly) assumed those would be kills and ranks since they're player specific, saved on mission outcome, and are not campaign specific.


I'm also confused by MageKing's terminology, what's player-specific supposed to mean compared to player-persistent? How are those 2 even different? I know what a specific variable is in math terminology but I don't understand how it would apply here since you aren't solving any equations.
« Last Edit: May 13, 2017, 10:28:54 am by FrikgFeek »
[19:31] <MatthTheGeek> you all high up on your mointain looking down at everyone who doesn't beam everything on insane blindfolded

 

Offline karajorma

  • King Louie - Jungle VIP
  • Administrator
  • 214
    • Karajorma's Freespace FAQ
Re: Persistent variables
The difference between the 1st and 4th is that the fourth one would persist even after you finish the campaign.

That means you could for instance make a variable called First_Playthrough and then have the campaign set up differently if you replay it. If you only have the first kind the second playthrough of the campaign would begin exactly the same as the first because all the variables would reset to their default values.
Karajorma's Freespace FAQ. It's almost like asking me yourself.

[ Diaspora ] - [ Seeds Of Rebellion ] - [ Mind Games ]

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Persistent variables
That's why I (wrongly) assumed those would be kills and ranks since they're player specific, saved on mission outcome, and are not campaign specific.
But they're not SEXP variables, so irrelevant to this conversation.

I'm also confused by MageKing's terminology, what's player-specific supposed to mean compared to player-persistent? How are those 2 even different? I know what a specific variable is in math terminology but I don't understand how it would apply here since you aren't solving any equations.
"Player-persistent" is the current name for what is actually still supposed to be a campaign variable. "Player-specific" is meant to be taken literally: a variable that is specific to the player (as opposed to being tied to the campaign).
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 
Re: Persistent variables
Maybe call them 'stashed' variables? Like RPG magic stashes that stay with your character no matter where they go. Though that might just add confusion for people not familiar with RPG terminology...
[19:31] <MatthTheGeek> you all high up on your mointain looking down at everyone who doesn't beam everything on insane blindfolded

 
Re: Persistent variables
This kind of terminology issue is why I suggested moving to a system of flags instead.

 

Offline 0rph3u5

  • 211
  • Oceans rise. Empires fall.
Re: Persistent variables
... how far a long is this now?

here is why I am asking:
Spoiler:
I frist became aware of this during a discussion how to hide a secret mission; right now I am considering to cut that mission from the initial release but to leave the system to access it in place hoping that a player-persistent variable can bridge the gap between to releases. (I always wanted that replaying the campaign to the requisiste for access and possibly functionality of the mission).

If the variable system is changed sooner rather than later, this is all for naught of course
"As you sought to steal a kingdom for yourself, so must you do again, a thousand times over. For a theft, a true theft, must be practiced to be earned." - The terms of Nyrissa's curse, Pathfinder: Kingmaker

==================

"I am Curiosity, and I've always wondered what would become of you, here at the end of the world." - The Guide/The Curious Other, Othercide

"When you work with water, you have to know and respect it. When you labour to subdue it, you have to understand that one day it may rise up and turn all your labours into nothing. For what is water, which seeks to make all things level, which has no taste or colour of its own, but a liquid form of Nothing?" - Graham Swift, Waterland

"...because they are not Dragons."

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Persistent variables
... how far a long is this now?
No work done, and we're still in a feature freeze so I have no real desire to start working on something this complicated right now.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline karajorma

  • King Louie - Jungle VIP
  • Administrator
  • 214
    • Karajorma's Freespace FAQ
:bump:

I'm thinking of starting work on persistence for SEXP Containers. That means I need a decision to have been made on this. Personally I still like the idea of two names and a flag for whether it is eternal or not. We could also add a flag for techroom use if required.

In terms of backwards compatibility, I'd suggest that anything currently in the player file becomes an eternal variable. As soon as the next mission is played, the code should notice that the eternal flag is missing and remove the variable from the player file (writing it into the campaign file instead). This does mean that any campaign which is counting on the bug being present would break but given that this can easily be fixed (simply re-release the campaign file with the eternal flag ticked) and given that knossos makes it easy to distribute the fix, I think that's the best solution.
Karajorma's Freespace FAQ. It's almost like asking me yourself.

[ Diaspora ] - [ Seeds Of Rebellion ] - [ Mind Games ]

 
How many mods rely on the bug, do you estimate?

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
I'd estimate very few; possibly even none, although that's always a dangerous guess regardless of circumstances.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline Goober5000

  • HLP Loremaster
  • Moderator
  • 214
    • Goober5000 Productions
I'm thinking of starting work on persistence for SEXP Containers. That means I need a decision to have been made on this. Personally I still like the idea of two names and a flag for whether it is eternal or not. We could also add a flag for techroom use if required.

In terms of backwards compatibility, I'd suggest that anything currently in the player file becomes an eternal variable. As soon as the next mission is played, the code should notice that the eternal flag is missing and remove the variable from the player file (writing it into the campaign file instead). This does mean that any campaign which is counting on the bug being present would break but given that this can easily be fixed (simply re-release the campaign file with the eternal flag ticked) and given that knossos makes it easy to distribute the fix, I think that's the best solution.

There are campaigns that are counting on the non-buggy behavior -- I know since I've released one or two -- so the bug ought to be fixed.

I like the two-axis, four-type concept, and I personally can't think of a better design, so I say go ahead and implement it.  I'm not sold on the terminology though.  AdmiralRalwood's explanation was succinct and clear, but "progressive", "eternal", and so forth don't intuitively match to each use case IMO.

How about global vs. campaign-local for one axis, and checkpointed vs. non-checkpointed for the other axis?