[Kamikaze]

Originally posted by CaptJosh


You think I use that trash? OpenOffice, man. All the way. And I'm talking text files over a megabyte in size, which is unusually large for the average spreadsheet or text doc.

And would you kindly quit complaining when someone is backing you up? Take it in the spirit in which it is intended. I swear, you can't help some people... [/B]

You're being quite rude to Kazan. What you posted was off topic and misinformed. This is not about word processing but about compiled code vs. interpreted code vs. bytecode.

Originally posted by karajorma

And you still haven't posted a single disadvantage.

I consider the disadvantage Kazan posted (needs a GUI to be effective) to be a major issue. A GUI can make the process unwieldly and inefficient. A scripting system would have the best of both worlds, coders could make script extensions and FREDers can keep doing their own thing. The scripters could give the FREDers more to work with too.

A scripting language also has more features "out of the box" that would need to be coded into a revamped sexp system (as mik said, it's like reimplementing LISP). Lua would give coders more flexibility than sexps because of its language features.

[Kazan]

it doesn't _need_ a gui to be efficient, that's just an easy-of-use tool - sexp structure is pretty simple to understand as you've seen me write sexps off the top of my head in this thread and my signature

there is no scripting language that overcomes the disadvantages it has when it comes to games - scripting languages are for non-realtime ops, and for web page serving, etc - not for realtime high performance applications.

[WMCoolmon]

Define me a new function with SEXPs.

Define me a new class with SEXPs

Do operations with floats with SEXPs

Show me a function that returns a value without messing up the conditional its a part of in SEXPs

Show me a way to implement functions without having to edit five separate values in different places in SEXPs

Show me the extensive modules in SEXPs that would let me do network operations

Show me the ability to expand the SEXP system via modules

Show me how I can multithread with SEXPs


There is a reason that Python is slower than the SEXP system. It provides a lot of things that the SEXP system doesn't.

In addition, I hear still here people talking like Python isn't being compiled into bytecode and then run. Python is being compiled into bytecode at program start. This bytecode is evaluated every frame. There is no parsing occuring every time Python is used. Past the first time python.tbl is parsed, the file is never opened again.

Kazan brought up a link I posted with Python: http://shootout.alioth.debian.org/benchmark.php?test=all&lang=python&lang2=gcc&sort=fullcpu

Although he obviously didn't read it or else he would be pointing out the 234x difference. (Apparently Python doesn't like extreme numbers of recursive functions). Which is something which shouldn't be happening.

Oh, and the tests are grossly inaccurate because there's no 'fully-featured SEXP' option. Python and SEXPs are both coded in C; I'm comparing raw C to Python in that example. It's not iterating through switch statements or cycling through nodes to figure out what the instructions are like SEXPs would be.

So to say that because SEXPs are coded in C and are therefore as fast as C is a gross fallacy.

AFAIK the benchmarks were run taking compile time into account. And, well, see above bolded text.


Also, python isn't intended to be a replacement for the SEXP system. I see it as being a step more than that; a way to perform lower-level operations on an individual mission or mod basis.


And just to reiterate reasons for dislike of the SEXP system, the postfix operator system is just another hurdle to overcome when learning to write with it. A GUI editor is unreasonable (IMHO) because it means that you're spending a lot of time sorting through menus even if you already know what functions you want to use. It has a number of problems and limitations, not the least of which is that it's frustrating to code functions for, no matter how simple.

Before it can be Python or Lua's equal, those have to be fixed. Then, IMHO, we can honestly argue processor time.

And finally, I'm not convinced that SEXPs are going to be so much incredibly faster than Python or Lua for any reason except their simplicity. Python is, after all, coded in C as well and was designed as a scripting language from the start, AFAIK, so it stands to reason it'd have a decent parser (compared to SEXPs which were implemented only to provide mission scripting).



Edit: Oh, and I have no confidence in this massive SEXP overhaul happening in the same amount of time...hell, Python's already in and has more abilities than the SEXP system, so go figure.
But I already asked about using SEXPs for the HUD system before, as I couldn't get a full expression system working and hadn't thought up using another scripting system, and that never happened. The overhaul that's being proposed is pretty major, and would take awhile to do & test & work all the bugs out of. There are over 100 SEXPs, each would have to be ported to the new system, for one...

[Kazan]

processor time is more important from the get go - we have four developers opposing and one in favor

i'd say "it's a no brainer" it's dead

[WMCoolmon]

I don't care what developers think. I'm more interested in what modders think.

Developers can just code features into the game source directly.

That's not to say I'm totally ignoring you guys. At least I hope not as I wrote a whole post responding to what you and Goober are talking about. I haven't contested that SEXPs are faster than Python because they probably are. I have absolutely no proof, I'm just making an educated guess as are you.

Mostly I think that a scripting language is better than trying to expand the SEXPs to duplicate the abilities of that scripting language.

Lua has a PS2 port going on. Now, it is possible that the people working on that are totally misguided and will find that the PS2 is totally unable to run a scripting language due to the "high" memory and CPU requirements. But I think they'd figure out whether or not it's feasible before embarking on such a project.

Why am I sticking with Python. Mostly because the alternative, upgrade the SEXP system to be as good as Python, is just reinventing the wheel. Python's already in. I can throw hooks wherever I like, granted in a somewhat hackish manner, but it's still there. I could implement vector and angle and matrix classes and let people replace the physics calculations with a Python script that would do the job.

Plus Python is built to be compiled on-the-fly; you could edit the physics calculations on the fly rather than recompiling them with VC++ every time.


I'm not familiar with Lua and nobody seems to care for it who doesn't want Python (even though it is used in a number of well-known games) so I'm not looking into it, although I did breeze through the documentation and it seems an attractive alternative.

[CaptJosh]

I guess I've failed to draw the comparison I was thinking of.

If a word processor, which is designed to open a text file and parse the text and formatting merely to display on screen, has a noticeable delay in loading a file that is as small as 1.5 MB, then think about how much more CPU work a script, or, more likely, a set of scripts might take? Scripts which must be parsed not merely to display formatted text on screen, but to execute some sort command set, recieve return from the execution, and move on through the script or scripts.

On second thought, perhaps a better analogy would be the difference between loading a copy of mIRC with a few aliases vs starting up a copy of mIRC that has a heavy set of scripts to load on startup, plus scripts that execute a lot of commands in the background while the program is running, and then going on to one of those HUGE mp3 channels on Undernet with all the warp scroll, having a huge back buffer for chat windows, and global chat logging on. Pretty soon, you start to notice things dragging.

[Kamikaze]

Originally posted by Kazan

processor time is more important from the get go

I'd argue that the programmer's time is more important than the processor's. I also trust the judgement of the professional game developers (such as Inquisitor, who seems sympathetic to WMC's cause) who use scripting languages on a daily basis to develop complex, real-time games.

Originally posted by CaptJosh
I guess I've failed to draw the comparison I was thinking of.

If a word processor, which is designed to open a text file and parse the text and formatting merely to display on screen, has a noticeable delay in loading a file that is as small as 1.5 MB, then think about how much more CPU work a script, or, more likely, a set of scripts might take? Scripts which must be parsed not merely to display formatted text on screen, but to execute some sort command set, recieve return from the execution, and move on through the script or scripts. [/B]

Perhaps it's best for you to start reading others' posts. Read this:

Originally posted by WMCoolmon

Python is being compiled into bytecode at program start. This bytecode is evaluated every frame. There is no parsing occuring every time Python is used. Past the first time python.tbl is parsed, the file is never opened again.

[CaptJosh]

I did read that. I was going back and trying to correct a misperception about what I was originally trying to say in an earlier post.


Just tack this on the end of what I last posted:

However, bytecode compilation on program start would seem to change things. How much it would change them, I'm not qualified to say.

[WMCoolmon]

I don't think anything you guys say is going to change my mind, since Python is wholly optional (It's optional to compile it in, it's optional to use scripts at all, either will result in basically nil performance difference).

I think there's immeasurable gain to be had right away from implementing a scripting system, far beyond what upgrading the SEXP system would do. The developers of the Unreal, Homeworld 2, Battlefield 2, Tribes 2/Torque game engines and many others agree. I think this gain is worth the increased processor power and memory usage. I think that, in the long term, upgrading the SEXP system to have all the features of one of these scripting language would be A) a waste of time since it's already possible via said scripting languages, and B) would cause an increase in processor and memory usage similar to what using a scripting language would.

I don't want to waste my time arguing about this anymore.

http://www.hard-light.net/forums/index.php/topic,35914.msg744514.html#msg744514

[WMCoolmon]

*Bump*

Status report...

I've been looking into getting Lua working in fs2_open and have it at roughly the same level of completion as Python.

In addition I've found this nifty little thing called LuaJIT, which has its counterpart in Python in the psyco project.

What these do is compile Python and Lua functions as machine code, as best possible. What this means is that it's approaching the level of being as fast as C, and/or SEXPs. I've got the LuaJIT library as part of my codebase right now, and working (except for the lua IO library), much like the OGG library.

The downside: They only work on x86. I'm not sure what'll happen on 64-bit or Macintosh processors.

[WMCoolmon]

Here's the C code for creating a ship, providing a ship object to python, and setting the speed of that ship object.

Code: [Select]

typedef struct lua_ship
{
ship *shipp;
} lua_ship;

static int lua_msn_newShip(lua_State *L)
{
char *ship_name = NULL;
char *class_name = NULL;
vec3d pos;
angles ang = {0,0,0};

if(!script_parse_args(L, "ssfff|fff", &ship_name, &class_name, &pos.xyz.x, &pos.xyz.y, &pos.xyz.z,
&ang.p, &ang.b, &ang.h))
return 0;

//New matrix
matrix ori = IDENTITY_MATRIX;
vm_angles_2_matrix(&ori, &ang);

//Find the class index
int si_idx = ship_info_lookup(class_name);
if(si_idx < 0)
{
//couldn't find it
return 0;
}

//Create the ship
int s_idx = ship_create(&ori, &pos, si_idx, ship_name);

//Make the lua ship object, if the ship was created
if(s_idx > -1)
{
//Make the lua ship object and set the ptr to NULL
lua_ship *lsp = (lua_ship*) lua_newuserdata(L, sizeof(lua_ship));
luaL_getmetatable(L, "ship");
lua_setmetatable(L, -2);
lsp->shipp = NULL;
lsp->shipp = &Ships[s_idx];
return 1;
}

return 0;
}

static lua_msn_ship_setSpeed(lua_State *L)
{
lua_ship *lsp;
vec3d vel;

if(!script_parse_args(L, "ufff", SCRIPT_UDATA("ship", lsp), &vel.xyz.x, &vel.xyz.y, &vel.xyz.z))
return 0;

//Set the speed
Objects[lsp->shipp->objnum].phys_info.vel = vel;

//We're done!
return 0;
}

static const script_func_list lua_msn_ship_funcs[] = {
{"setSpeed", lua_msn_ship_setSpeed},
{SCRIPT_END_LIST},
};

//LIBRARY define
static const script_object_list lua_msn_lib_obj[] = {
{"ship", lua_msn_ship_funcs},
{SCRIPT_END_LIST},
};
static const script_func_list lua_msn_lib[] = {
{"newShip", lua_msn_newShip},
{SCRIPT_END_LIST},
};

That all makes up the Mission (abbreviated to "msn") library. So to add that, plus the grl and base libraries...

Code: [Select]

static const script_library_list Lua_libraries[] = {
{"grl", lua_grl_lib, NULL},
{"msn", lua_msn_lib, lua_msn_lib_obj},
{NULL, lua_base_lib},
{SCRIPT_END_LIST},
};

I've set it up so that the init functions will take those and run the required Lua functions on them.

As a result you can do this from Lua script:

Code: [Select]

my_ship = msn.newShip("GTD Rampart", "GTD Orion", 10, 15, 20)
my_ship:setSpeed(0, 0, 3000)

Well, with one small hitch; for some reason, the Lua 'ship' object isn't getting set as the right userdata type.

[Goober5000]

Originally posted by WMCoolmon
Mission (abbreviated to "msn")

Oh noes!

Is abbreviation necessary?  If so then maybe you should call it fsm instead.

[WMCoolmon]

Yeah, it's funny how standard coder abbreviations work out to names that sound bizarre (f y cn rd ths, y shld b a prgrmmr). EG "graphics local" = grl = girl, "mission" = msn

For *final* lib names, I was thinking using four-letter words (no, not those four-letter words) for all the global libs; three-letter words would be used for local definitions. Object types would keep their full name to keep total obfuscation from happening. (Except for maybe "jump node"..."node" seems to work nicely...yes...)

[Bobboau]

Originally posted by Kazan
we have four developers opposing and one in favor

two in favor.
it'd be nice to have the option.

[WMCoolmon]

Took another look at this.

I was able to get the class working properly, except Lua crashes shortly after setting the ship speed.

On the plus side, it looks like once you have a class, it's easy to define array indexing functions. So...

Lua

Code: [Select]

Ship["hull"] = 1000

C

Code: [Select]

static int lua_msn_ship_elem_set(lua_State *L) {
char* var_name;
float* new_hull;
lua_ship* lsp;
if(!script_parse_args("usf", SCRIPT_UDATA("ship", lsp), var_name, new_hull)) {
     return 0;
}

if(!stricmp(var_name, "hull")) {
     lsp->shipp->hull_strength = new_hull;
}
return 0;
}

//Also, add it to the object functions
static const script_func_list lua_msn_ship_funcs[] = {
{"set", lua_msn_ship_elem_set, SCRIPT_GET_FUNC}, //<-- this one
{"setSpeed", lua_msn_ship_setSpeed, NULL},
{SCRIPT_END_LIST},
};


That's all educated guess right now; I need to break apart one of the Lua auxlib functions before I can do it like that.

I think the crash may be caused by LuaJIT, unfortunately. I figure I'll try compiling with the vanilla lua libs and see if the crash still occurs. I've already had LuaJIT reliably crashing when I try to load the Lua IO library...

On the plus side, I seem to have managed to get precompiled functions working.

Edit: Removal function guess:

Code: [Select]

lua_pushstring(Ssn, lib_name);
lua_gettable(Ssn, LUA_GLOBALS_INDEX);
lua_pushstring(Ssn, func_name);
lua_gettable(Ssn, -2);
lua_pop(Ssn, -1);

[WMCoolmon]

BTW if anyone has some opinions on general syntax or libraries or functions that should be included, now's the time to air them. The three major items on the to-do list are:

• Get object-oriented programming working (without consistent crashes)
• Get LUA working as an expression system
• Figure out how to remove functions/libraries/objects once they're added.

[Nuke]

how big will out list of game data variables be? also the ability to access any pof data of a model, and to get coorinates in absolute and relative coordinates (relitive would be usefull if newtonian ever gets implemented). like if i need to know where a point in a dock path is, i can get it in absoluute coordinates, or relitive to any ship in the mission.

[Bobboau]

stuf like that would almost exclucively be in local (relitive to the object it's on) coordanants, though a global_space(vector) function would probly be easy.

[WMCoolmon]

Hmm, how about making a special local vector and separate world vector classes?

Or, making a vector class, special local class, and special world class; the vector class would hold the vector in local and world coordinates, the two 'special' classes would be gotten by calling a function to return them.

Or () using specific names for functions, eg "grl" for local graphics, "grw" for global-vector based graphics.

The list of game data variables would be as big as it needs to be. For the functions currently provided, at least major ship and weapon variables.

[Nuke]

access to pof data would be good to have as well. id like at some point to be able to script custom sensor modes as well as hud gauges. also would there be a way to implement custom keyboard comands via scripting that would actually appear in the control config.