[Wanderer]

Something suggested by blowfish...

Should Flashy Deaths script be included to the code? As some people seemed to think it might be useful and easier to handle as a coded version. That is it uses separate data files and has rudimentary parser. Scripting probably adds some overhead so it presumably would run faster as a c/c++ coded version. Also direct code integration would allow for more accurate controls for certain effects.


TBH I'm not terribly fond of the idea but then again I'm really not against it either.

[Echelon9]

My view is that whatever can be done reasonably with scripting should be done in scripting, leaving the engine C/C++ code as an interface to the engine's exposed features.

There is a burden associated with increasing the size of the C/C++ codebase, and it would appear at present that this feature can be implemented adequately in scripting.

[chief1983]

But like he said, performance of this feature is probably not the best in scripting.  I'm kind of the opposite, a lot of stuff that can be done in scripting is very clunky, and support for it in the engine would make things a lot smoother sometimes.

[Tomo]

I would say yes - it should be moved to code.

It's a graphics-intensive operation, and is designed to be configured by a table file. (Albeit currently a slightly different format)

- It would fit nicely into fireball.tbl, as modifiers to custom fireballs.

[Nuke]

i see a lot of script used for prototyping, and then having working stuff re-written in c to improve performance. like that particle thing i did awhile back. if one looks at the numer of tables a modern mod has compaired to a retail mod, youl notice a major increase in the amount of stuff. you could call it bloat. scripting can solve this, but i think it needs to be streamlined more and made more user friendly.

frequently used systems, like parsing and metadata and input so far seem like major areas for improvement. these three systems are probably more than half of the lua used in fsrts, atmospheric flight, and advanced cockpits scripts. i usually include these systems when i start a new scripting project. if they were a part of the scripting system it would improve the system greatly and reduce the amount of lua we need to write to add a basic feature.

parsing and metadata could have their interfaces combined. meta pertaining to a ship class could be parsed directly from the ships.tbl and would be accessible via the shipclass handle. meta pertaining to a specific ship could be parsed from a mission file or created at init and modified at runtime. current meta systems have to make frequent calls to a scripted lookup function that looks for a ship with a specific object signature, if instead i could just look up this data directly in the handle to that object, it would make things much faster. most scripts with gameplay features seem to revolve around object derivitaves and their handles, and often require static class data (like how fast this kind of object can move or turn) and dynamic object instance data (how fast the particular object is moving or turning), while most of this data is available through handles, it doenst do much for you if the data is something that you need to define. essentially you should be able to stick an arbitrary piece data into the handle, and then retrieve it, modify it, put it back in, and look at it in some other part of the code, ect. by integrating the meta system in this way, it would take care of both meta and parsing.

input could use the greatest upgrade. the big issue is that input needs to be abstracted down from direct access meathods we currently have (key hooks, mouse hooks, controls library functions), which are directly associated to a physical button or mouse axis, and reduce it to a command level interface, where the control that is bound to the command is irrelevant. within the script, if you wanted to check on the state of a command, you could look that command up as a key in a lua table, and the value would indicate the control state for that command (returning a number for axis commands and a bool for button/key commands). commands should eventually be bindable through the existing control options screen. support for command aliasing/removing/renaming/addition should also be a part of the system.

[DerKuchen]

I don't have any experience with FSO Scripting, but there are some general things to make a Lua script faster:

1. When possible use local variables, a global lookup involves at least one table (and maybe a metatable) lookup where a local variable can be accessed (nearly) immediately.
2. Make local references to often used global functions/tables.
3. When possible don't create tables in inner loops or very often called functions (say every frame or at a small timer interval). The Lua 5.1 GC is good, but many small objects may have a performance impact.

A great resource for performance tips in Lua is this PDF: http://www.lua.org/gems/sample.pdf

Looking through the 'Flashy Deaths' script, wouldn't it be faster to use Lua for the *.cfg files too? Parsing them would be mainly a matter of dofile or loadstring.
And I think making local references to e.g. math.random and some of the often used global tables (arr_*_eff) in the OnFrame section of the code could speed it up a bit.

[Wanderer]

Thank you for the input as well as for the advice and the link...

And yes, making the cfg files into lua would be a lot faster but speed is not of the essence in that section. That is once the date is loaded there is no more need to access the file. Also in early script versions users didn't like to input values into lua files but wanted the data input to be easier and exists outside the .lua files, though this may have changed by now.

And yeah... i havent had time to go through the flashy deaths script in detail. There are still sections in it which could use streamlining (i personally hate the one section of the parsing code, probably should made the currently parsed lined into real global in the first place instead of joggling it back and front the function calls. And i am certain several local flags could be added to the script.

[Nuke]

parsing in lua really doesnt hurt ingame performance. my tbl parser can be inserted into a script (dofile), run once in an init hook, and then spit out a hierarchy of tables you can use elsewhere in the code. since tables are references this is a very fast way to do things. i used to think dofile didnt work in this implementation of lua, but apparently it requires a path relative to fsroot, and not the mod dir. this isnt such a bad thing though, since it allows you to put modular scripts in one place. , but you kinda have to not change the mod dir from what is used in your dofile paths if you wnat to run files from the local mod dir. perhaps make mod path and root path strings global variables. that way you dont have to use a hardcoded mod folder name.

the meta system im using is fairly fast (and it better be after rewriting it half a dozen times). the first system did a lot of table creation in loops to find new objects and delete removed ones from the metasystem. i had used the ship name as they key but after several attempts to streamline it, i decided i needed a unique identifier that could apply to all objects in the game, not just ships. so i added a new functions in c to get an objects unique numeric signature from within the engine, as well as a function to get an object handle from this signature. now i just use the number as the key. and now the only loop in the whole system is only used to delete dead entries every few seconds.

[chief1983]

It's probably relative to the engine and not the game data root then.  If we start trying to put the engine away from the game data root that could still cause issues.

[Nuke]

we could add something like ba.getEngineRoot() or ba.getModRoot() to our base library which would return the proper path string. then when you needed to use a dofile, you could just use:

dofile(ba.getModRoot().."data/scripts/something.lua")
instead of
dofile("modname/data/scripts/something.lua")

for simplicity you may want to also include functions to get to your scripts root or your table root, of course these could be setup as strings at init, and that would probibly be faster. i kinda dont want to loose the ability to put scripts in a global directory, which is very useful for development purposes. you can make a modular script and test it against other scripts, which would make debugging it difficult if every script you were working on used a local copy.

[Echelon9]

we could add something like ba.getEngineRoot() or ba.getModRoot() to our base library which would return the proper path string. then when you needed to use a dofile, you could just use:

dofile(ba.getModRoot().."data/scripts/something.lua")
instead of
dofile("modname/data/scripts/something.lua")

I see real benefit in these two additional functions. A mod author can never be quite sure that the user installed the mod in a folder with exactly the same name as used while developing or as recommended in a manual.

It would also give benefits to those total conversion developers who initial develop as a mod, and then need a quick way to scan+replace as they move to a top level TC file structure for release.

[chief1983]

My point is that I don't think the engine should care where _it_ is.  I've been talking with the SCP about the possibility of making it so the exe does not have to exist in the folder with the game content at all.  It would simply need to know where the game content is, which could be defined in a config file in the appdata or userdata folder.  We've got a lot of work to do in this area but it really needs to be done to make the engine more cross-platform compliant, on all platforms.

[Zacam]

Agreed.

[Wanderer]

Nuke.. Why wont just use cfile and let it do the file locating for you?

[Nuke]

just because it shouldnt care where it is, doesnt mean it that it does not need to know where it is. scripting kinda also needs to know where it is, then scripters and modders dont need to make any guesswork as to where lua files are locted, just call a function for the correct path. ideally every lua function should work relative to the mod dir, but there are some built in functions, like dofile, that like to run relative to the program root. so this kinda helps meet that goal.

Nuke.. Why wont just use cfile and let it do the file locating for you?

dofile is equivalent to an include call in c, it has a completely different usage from cfile. dofile helps make script more modular and for large scripts, much more organized.

[chief1983]

I'm saying, we may try to move the engine outside of the game data folder, at least on some platforms.  You won't be able to rely on the program's own folder being the same as the game's main data folder or the parent of the mod folders.  It probably won't affect what your talking about as long as this is taken into account.

[Tomo]

One thing you should know about FS2 Open and Mods:

FS2 Open does not know which mod you are running - and it's impossible to find out.
This probably surprises you, but it's true.

The Launcher passes a list of mod folders to FS2 Open, which may be a single folder or could be several folders. The actual chosen Mod is often the first one in the list - but not necessarily.

FS2 Open also handles files (including tbls and tbms) in such a way that it would be impossible for it to pass a file location to Lua - the file could be in a VP or in the real filesystem.

It would make much more sense for FS2 Open to generically parse a set of tables and put it straight into the Lua datastructure.
- This also means that any changes to how FS2 Open finds the files wouldn't break Lua.

However, that is definitely a lot of work.

[Nuke]

fortunately everything you need to write a parser defaults to the local mod dir, so thats not the issue. the biggest issue is when your script gets very complicated and you need to cut each hook up into more than one lua file. as i said dofile is like an include in c, but it runs in relation to the executable path. its part of the lua core library so it associates its path with that of the executable. if getting mod dir path data for dofile is impossible, then a substitute must be added.

the atmospheric flight script is about 900 lines of code plus another 400 for my tbl parser, it is also not my largest script. because of this i dont put the lua directly in the scripting.tbl (or xxx-sct.tbm). so my scripting.tbl looks roughly like this:

Code: [Select]

#Global Hooks
$GameInit:
[[atmospheric_flight_gh_init.lua]]
$Simulation:
[[atmospheric_flight_gh_sim.lua]]
$Hud:
[[atmospheric_flight_gh_hud.lua]]
#end

#Conditional Hooks

$State: GS_STATE_GAME_PLAY
$On Key Pressed: [[keydn.lua]]
$On Key Released: [[keyup.lua]]

$Application: FS2_Open
$On Mission Start:
[[on_mission_start.lua]]

$Application: FS2_Open
$On Mission End:
[[on_mission_end.lua]]

#End

with all the lua files being in moddir/data/scripts/. there are 2 reasons i do this. the main one being debug purposes. when an error is reported at line x, and it is in a lua file, then that line number is exactly where the problem is (because line 1 corresponds to the first line in the file). if the script was in the table file then it would consider line 1 as the first line of script in the hook, which may actually be line 900 in the text file. this makes finding the problem line really complicated. the other reason i keep my lua files seprate is to organize my code. you can also use an ide with the lua files and a text editor with your tbl files without getting your file associations confused.

now in a perfect world dofile() would default to that moddir/data/scripts/, and it might be possible to tell the lua api to use this dir at init. however if that is not the case, it would become very convenient to be able to list lua files in the table, so you could do something like:

Code: [Select]

#Global Hooks
$GameInit:
[[parser.lua]]
[[metasystem.lua]]
[[dynamics.lua]]
[[gauges.lua]]
$Simulation:
...

using a tbm for this kinda thing doesnt work either. if the script in the tbl depends on code in the tbm, theyre may be errors. tbms should mostly be used in situations where the scripts are unrelated and one doesnt depend on the other to run error free, usually for mini-scripts that add a single simple feature. for example, running the escort retical with a velocity gauge.

to avoid confusion, all the other ideas i posted for streamlining scripting (built-in parsing, meta, and input abstraction) were mainly ideas for streamlining script, and really have nothing to do with the problem of code organization and path issues.

[Iss Mneur]

Nuke, I think you missed what was meant by "using cfile". cfile in the context of FSO is not the C file api (ie. the ISO standard functions for File IO; fopen, fwrite, etc.) but but the name of a code file "cfile.cpp" that contains all (with very few exceptions, lua being one) methods of accessing the filesystem in the engine.  cfile.cpp it what the argument of the -mod flag is passed to, and is what allows the engine its ability to be modded to the extent that it can.  It is also because of cfile that FSO does not know or even care what mods are actually passed via the -mod flag.

FSO's cfile api allows code anywhere in the engine to load a file by just passing the files name and cfile will search the folder that corresponds to its extension in the first listed mod folder, then every .vp in the first listed mod folder that has the folder that corresponds to the extension, this is repeated for every mod folder listed, then for the root folder.  For example, say -mod mod1,mod2,mod3 is passed to FSO and that you requested atmospheric.lua, cfile will search inorder:

• mod1/scripts/atmospheric.lua
• mod1/vp1.vp:data/scripts/atmospheric.lua
• ...
• mod1/vpn.vp:data/scripts/atmospheric.lua
• mod2/data/scripts/atmospheric.lua
• mod2/vp1.vp:data/scripts/atmospheric.lua
• ...
• vp1.vp:data/scripts/atmospheric.lua
• ...
• vpn.vp:data/scripts/atmospheric.lua
• data/scripts/atmospheric.lua

until it finds a file that exists.  This behaviour is also what allows for the mixing of game assets, like blue planet and their layering of the ships to have new designs but still have the stock FS2 and FSU assets. Even used this principle of cfile to allow for normal effects and advanced effects to be activated by adding a .vp (the name of which escapes me) that is earlier in lexical sorting, so that cfile will search and find the high end version of a graphic in the new .vp rather than using the low end one in the Root.vp.

just because it shouldnt care where it is, doesnt mean it that it does not need to know where it is. scripting kinda also needs to know where it is, then scripters and modders dont need to make any guesswork as to where lua files are locted, just call a function for the correct path. ideally every lua function should work relative to the mod dir, but there are some built in functions, like dofile, that like to run relative to the program root. so this kinda helps meet that goal.

No, this is where the problem lies, the script should not care nor should it know where it is located on the filesystem.  As I described above, lua is circumventing how the entire engine finds all of its resources.  The built-ins that do not use cfile should either be modified to use cfile or removed and replacements implemented which will use cfile.  Using cfile does make the scripts more modifiable and flexible.

Using cfile allows anyone to do mods based on your scripting work by just putting your mod in the -mod list and having their work before yours, that way they could change the interface or the ship models, etc.

Nuke.. Why wont just use cfile and let it do the file locating for you?

dofile is equivalent to an include call in c, it has a completely different usage from cfile. dofile helps make script more modular and for large scripts, much more organized.

From your description, Nuke, it seems you are trying to use dofile as a replacement for function calls. Is there something wrong with the function calls in lua? FSO's lua?

In short, I do not mean to call you out Nuke, but why should lua be exempted from the resource location system that the rest of the engine has?

[portej05]

The other issue is that if the mod system changes the way it works, you run into a lot of trouble, and won't get any support from it.