Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: ARSPR on May 19, 2007, 01:16:01 pm

Title: Several questions about tbm and tbl
Post by: ARSPR on May 19, 2007, 01:16:01 pm
Because of these comments (http://www.hard-light.net/forums/index.php/topic,46817.msg958067.html#msg958067) from CP5670 in his PI release thread, I've started doubting about what I thought I knew about the way tbm and tbl tables work. I've checked Wiki but there's no clear info about it.

Although I've made several tests, (with a test-shp.tbm which changes the pof model for Ulysses), I would like some coder to confirm them. I will update wiki afterwards because it can be hugely important info for modders.

Code: [Select]
#Ship Classes
#End

(Maybe if the last one is fully right, it could be worthy to make FS2 allow full empty tbm files as a way to "delete" any other existing tbm with the same name).

Thanks in advance.
Title: Re: Several questions about tbm and tbl
Post by: Wanderer on May 19, 2007, 01:34:02 pm
Well

2) TBMs are parsed after the actual tables. So 'directory structure' based hierarchies do not apply

3) Yes

4) All tbms get loaded. In alphabetical order. So if you had aa-shp.tbm and bb-shp.tbm then game would first use values from aa-shp.tbm and then from bb-shp.tbm.

5) Have to use a 'blank tbm' of the exact same name.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 19, 2007, 02:38:35 pm

4) All tbms get loaded. In alphabetical order. So if you had aa-shp.tbm and bb-shp.tbm then game would first use values from aa-shp.tbm and then from bb-shp.tbm.

Ooops, I've re-made some more tests and it seems there's something wrong. (a bug?). I'm always using xxxxx-shp.tbms with the next shape:
Code: [Select]
#Ship Classes
$Name: GTF Ulysses
+nocreate
$POF file:      xxxxxxx.pof
#End
where  I've checked that xxxxxxx.pof DOES exist.

and sometimes the "blank" xxxxx-shp.tbm with the next shape:
Code: [Select]
#Ship Classes
#End


+ If I just put aa-shp.tbm (fighter03.pof - Dragon) and zz-shp.tbm (fighter2t-02.pof - Herc II) in freespace2\data\tables, I get Herc II instead of expected Dragon (????)

+ If in this situation, I add a zz-test.vp which has a "data\tables\prueba-shp.tbm" (fighter2t-01.pof - Pegasus), then I get Pegasus instead of any of the others.

+ If in this situation, I add my "blank" xxxx-shp named prueba-shp.tbm to freespace2\data\tables then I get
again Herc II.

So it seems that:
+ Files are loaded in standard priority and alphabetical order. Files in higher priority places override files in lower priority places. In this way, "blank" prueba-shp.tbm overrides the vp'ed prueba-shp.tbm
+ But then, parser starts applying changes from the highest priority file to the lowest one. This results in that the lowest priority file entries in fact have the highest priority.

I think you should change the latest behaviour because can lead to unexpected situations...
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 19, 2007, 10:56:29 pm
TBM's are loaded in reverse priority order, so the lowest priority order should be loaded first and the highest priority folder should be loaded last (to override anything that came before it).  I say "should" there, since the code is actually broken and it doesn't get sorted properly.  And ... prepare yourselves ... I'm the one that broke it ... 3 years ago.  :rolleyes:

Anyway, yeah, bug fixed.


EDIT:  And just to make sure it's understood: the files are priority sorted only, not sorted alphabetically (excluding the fubar from the code bug).  They are only sorted in the reverse order they are found.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 20, 2007, 04:26:06 am
TBM's are loaded in reverse priority order, so the lowest priority order should be loaded first and the highest priority folder should be loaded last (to override anything that came before it).  I say "should" there, since the code is actually broken and it doesn't get sorted properly.  And ... prepare yourselves ... I'm the one that broke it ... 3 years ago.  :rolleyes:

Anyway, yeah, bug fixed.

How have you been able to do that!!!??  You deserve 1000 lashes and execution ... !!!!  ;)


Well, I think this is the explanation to what CP5670 says in that thread:

Quote
However, it looks like something was changed with the game's behavior at some point, since it used to be different when I tested it out about two years ago. I remember that I was unable to override anything in the media VPs unless I did it this way, which included the stats in the weapon selection screen and some other things. If this has changed at some point, it makes things rather more convenient.

IMO, this is a so serious bug you should launch an official release, (let's say 3.6.10), ASAP even without any more additions or changes. Maybe a lot of strange behaviours can be caused by this one, (as example, I just remember DaBrain telling me that thrusters didn't show up as they should in one test with his new media vps WIP)

(So your Xt builds will become 3.6.11). Oops, this is going to be offtopic...


An then, only one of my questions keeps unanswered and tables "worked" as I expected. The XTM one. Any light to my darkness, please?

EDIT ------------------------------------

And also, please consider this suggestion about blank overriding tbms:

(Maybe if the last one is fully right, it could be worthy to make FS2 allow full empty tbm files as a way to "delete" any other existing tbm with the same name).

ie, make parser just ignore empty tbms without giving any errors, (or giving only a warning in debug build).

Title: Re: Several questions about tbm and tbl
Post by: taylor on May 20, 2007, 06:20:06 am
It's been broken since July of '04, less than two months after TBM support was originally added, so I doubt that anyone actually remembers it working any differently.  Even when it did work correctly, it was added in such a way that it broke several other things.  It just happens that the fix I made managed to break that particular sort order in the process. :)

3.6.10 will be released in about 2 months though, everyone can wait that long.

An then, only one of my questions keeps unanswered and tables "worked" as I expected. The XTM one. Any light to my darkness, please?
I don't really get the whole XTM difference, so that's probably something for WMCoolmon to answer.

And also, please consider this suggestion about blank overriding tbms:
It already supports blank tbms, for *-shp.tbm at the very least.  It just can't be zero byte.  Add a single space, blank line, or tab in the file and it should work just fine.  It wouldn't for for something like *-fbl.tbm, as that one has a very particular layout.  It should work for *-shp.tbm, *.wep.tbm, and *.amr.tbm though.  Getting *-fbl.tbm to support that as well shouldn't be too difficult, if it's needed, but someone will have to point out any TBM which doesn't work before we try and fix it.

Just remember that the new FS2NetD code that will be in 3.6.10 is going to support TBMs, so you'll have to deal with CRC validation issues.
Title: Re: Several questions about tbm and tbl
Post by: WMCoolmon on May 20, 2007, 02:17:50 pm
Because of these comments (http://www.hard-light.net/forums/index.php/topic,46817.msg958067.html#msg958067) from CP5670 in his PI release thread, I've started doubting about what I thought I knew about the way tbm and tbl tables work. I've checked Wiki but there's no clear info about it.

Although I've made several tests, (with a test-shp.tbm which changes the pof model for Ulysses), I would like some coder to confirm them. I will update wiki afterwards because it can be hugely important info for modders.

  • Which is the difference between a XTM tbm (example: ships.tbl and its -shp.tbm) and a non-XTM tbm (armor.tbl and its -amr.tbm)?. The explanation in Wiki isn't clear at all, (at least for me  :wtf: ).

XMTs:
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 21, 2007, 03:03:49 pm
Thanks WMCoolmon.

I think I've discovered another bug, but as I'm just starting playing with a lot of this stuff I prefer posting here than Mantising.

I'm using official 3.6.9. build.

I've tried to convert new media vps stars.tbl in a mv_adveffects-str.tbm. So I just have erased redundant entries, ($Bitmap: dneb01 and so on), from stars.tbl. As it is a non-XTM tbm table I've repeated all entries for suns and debris.

So my new mv_adveffects-str.tbm is:

Code: [Select]
; background bitmaps - random stuff
; $Bitmap is for a bitmap which should use the intensity == alpha blending mode
; $BitmapX is for a bitmap which should use 0, 255, 0 == transparent, and no blending otherwise


$Bitmap: nebulA1-main
$Bitmap: nebulA2-main
$Bitmap: nebulA3-whisp
$Bitmap: nebulA4-whisp
$Bitmap: nebulA5-whisp
$Bitmap: nebulA6-whisp
$Bitmap: nebulA7-whisp
$Bitmap: nebulA8-whisp
$Bitmap: nebulA9-whisp
$Bitmap: nebulA10-whisp
$Bitmap: nebulA11-whisp
$Bitmap: nebulA12-whisp
$Bitmap: nebulA13-whisp
$Bitmap: nebulA14-whisp
$Bitmap: nebulB1-main
$Bitmap: nebulB2-main
$Bitmap: nebulB3-main
$Bitmap: nebulB4-whisp
$Bitmap: nebulB5-whisp
$Bitmap: nebulB6-whisp
$Bitmap: nebulB7-whisp
$Bitmap: nebulB8-whisp
$Bitmap: nebulB9-whisp
$Bitmap: nebulB10-whisp
$Bitmap: nebulB11-whisp
$Bitmap: nebulB12-whisp
$Bitmap: nebulB13-whisp
$Bitmap: nebulB14-whisp
$Bitmap: nebulC1-main
$Bitmap: nebulC2-main
$Bitmap: nebulC3-whisp
$Bitmap: nebulC4-whisp
$Bitmap: nebulC5-whisp
$Bitmap: nebulC6-whisp
$Bitmap: nebulC7-whisp
$Bitmap: nebulC8-whisp
$Bitmap: nebulC9-whisp
$Bitmap: nebulC10-whisp
$Bitmap: nebulC11-whisp
$Bitmap: nebulC12-whisp
$Bitmap: nebulC13-whisp
$Bitmap: nebulC14-whisp
$Bitmap: nebulD1-main
$Bitmap: nebulD2-main
$Bitmap: nebulD3-whisp
$Bitmap: nebulD4-whisp
$Bitmap: nebulD5-whisp
$Bitmap: nebulD6-whisp
$Bitmap: nebulD7-whisp
$Bitmap: nebulD8-whisp
$Bitmap: nebulD9-whisp
$Bitmap: nebulD10-whisp
$Bitmap: nebulD11-whisp
$Bitmap: nebulD12-whisp
$Bitmap: nebulD13-whisp
$Bitmap: nebulD14-whisp
$Bitmap: nebulE1-main
$Bitmap: nebulE2-main
$Bitmap: nebulE3-main
$Bitmap: nebulE4-whisp
$Bitmap: nebulE5-whisp
$Bitmap: nebulE6-whisp
$Bitmap: nebulE7-whisp
$Bitmap: nebulE8-whisp
$Bitmap: nebulE9-whisp
$Bitmap: nebulE10-whisp
$Bitmap: nebulE11-whisp
$Bitmap: nebulE12-whisp
$Bitmap: nebulE13-whisp
$Bitmap: nebulE14-whisp
$Bitmap: nebulF1-main
$Bitmap: nebulF2-main
$Bitmap: nebulF3-whisp
$Bitmap: nebulF4-whisp
$Bitmap: nebulF5-whisp
$Bitmap: nebulF6-whisp
$Bitmap: nebulF7-whisp
$Bitmap: nebulF8-whisp
$Bitmap: nebulF9-whisp
$Bitmap: nebulF10-whisp
$Bitmap: nebulF11-whisp
$Bitmap: nebulF12-whisp
$Bitmap: nebulF13-whisp
$Bitmap: nebulF14-whisp
$Bitmap: nebulG1-main
$Bitmap: nebulG2-main
$Bitmap: nebulG3-main
$Bitmap: nebulG4-whisp
$Bitmap: nebulG5-whisp
$Bitmap: nebulG6-whisp
$Bitmap: nebulG7-whisp
$Bitmap: nebulG8-whisp
$Bitmap: nebulG9-whisp
$Bitmap: nebulG10-whisp
$Bitmap: nebulG11-whisp
$Bitmap: nebulG12-whisp
$Bitmap: nebulG13-whisp
$Bitmap: nebulG14-whisp
$Bitmap: nebulH1-main
$Bitmap: nebulH2-main
$Bitmap: nebulH3-main
$Bitmap: nebulH4-whisp
$Bitmap: nebulH5-whisp
$Bitmap: nebulH6-whisp
$Bitmap: nebulH7-whisp
$Bitmap: nebulH8-whisp
$Bitmap: nebulH9-whisp
$Bitmap: nebulH10-whisp
$Bitmap: nebulH11-whisp
$Bitmap: nebulH12-whisp
$Bitmap: nebulH13-whisp
$Bitmap: nebulH14-whisp
$BitmapX: planeta1
$BitmapX: planetb
$BitmapX: planetc
$BitmapX: planetd
$BitmapX: planete
$BitmapX: planetf
$BitmapX: planetg
$BitmapX: planeth
$BitmapX: SunSathanas01
$BitmapX: SunSathanas02
$BitmapX: SunSathanas03

#end

; sun bitmaps and lights - NOTE all $Suns must have a $Sunglow and a $SunRGBI!

$Sun: SunWhite
$Sunglow: Sunglow01
$SunRGBI: 1.0 1.0 1.0 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

$Sun: SunRed
$Sunglow: SunglowRed
$SunRGBI: 1.0 0.1 0.1 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

$Sun: SunGreen
$Sunglow: SunglowGreen
$SunRGBI: 0.5 1.0 0.5 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

$Sun: SunBlue
$Sunglow: SunglowBlue
$SunRGBI: 0.2 0.2 1.0 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

$Sun: SunGold
$Sunglow: SunglowGold
$SunRGBI: 0.74 0.76 0.44 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

$Sun: SunViolet
$Sunglow: SunglowViolet
$SunRGBI: 0.66 0.44 0.54 1.0
$Flare:
+FlareCount: 6
$FlareTexture1: corona001
$FlareTexture2: corona002
$FlareTexture3: corona003
$FlareTexture4: corona004
$FlareTexture5: corona005
$FlareTexture6: corona006
$FlareGlow1:
+FlareTexture: 1
+FlarePos: 0.0
+FlareScale: 5
$FlareGlow2:
+FlareTexture: 2
+FlarePos: 0.1
+FlareScale: 16
$FlareGlow3:
+FlareTexture: 3
+FlarePos: 0.3
+FlareScale: 2
$FlareGlow4:
+FlareTexture: 4
+FlarePos: 0.6
+FlareScale: 1
$FlareGlow5:
+FlareTexture: 5
+FlarePos: 1.2
+FlareScale: 6
$FlareGlow6:
+FlareTexture: 6
+FlarePos: 1.8
+FlareScale: 4

#end

; debris .ani's for NON-nebula mode. NOTE : there must always be 4 of these
$Debris: debris01
$Debris: debris02
$Debris: debris03
$Debris: debris04

#end

; debris .ani's for nebula mode. NOTE : there must always be 4 of these
$DebrisNeb: gas
$DebrisNeb: gas
$DebrisNeb: gas
$DebrisNeb: gas

#end

#end

Probably you could test with a simplified version (take out all LS Nebulas and sun glares), but I prefer posting all info.

Then when launching FRED (I want to test I've got all available new nebulas in background editor), I get the following error:

Code: [Select]
Error: mv_adveffects-str.tbm(line 353:
Error: Missing required token: [$Debris:]. Found [debris01]  instead.

File:J:\src\cvs\fs2_open_3_6_9.final\code\Parse\PARSELO.CPP
Line: 659
[This filename points to the location of a file on the computer that built this executable]

Call stack:
------------------------------------------------------------------
    fred2_open_d.exe 0050b3e9()
    fred2_open_d.exe 004fe5e8()
    fred2_open_d.exe 00510758()
    fred2_open_d.exe 004fee0c()
    fred2_open_d.exe 00455be7()
    fred2_open_d.exe 0044c951()
    fred2_open_d.exe 009ba792()
    fred2_open_d.exe 009ba0f4()
    fred2_open_d.exe 009b7c49()
    fred2_open_d.exe 009b8135()
    USER32.dll 7e398734()
    USER32.dll 7e39d05b()
    USER32.dll 7e39b4c0()
    USER32.dll 7e39f9fe()
    ntdll.dll 7c91eae3()
------------------------------------------------------------------
Title: Re: Several questions about tbm and tbl
Post by: Turey on May 21, 2007, 04:53:18 pm
I'm fairly sure the cause of that problem is here (starfield.cpp:~1276):
Code: [Select]
if (Parsing_modular_table && !optional_string("$Debris:")) {
break;
} else {
required_string("$Debris:");
}

The problem is that when optional_string returns true, it moves the read point past the "$Debris:". Since there's a not on it, it then becomes false, and we enter the else statement. It then checks for "$Debris:", finds the name of the debris instead, and errors out.

There's similar faulty logic a few lines afterwards. Both can be fixed by replacing optional_string with check_for_string, which doesn't advance the read point.

EDIT: Note that, due to short-circuiting, this only causes problems with modular star tables. Since few people ever touch stars.tbl at all (much less make a tbm of it), it's easy to see how the bug stayed hidden.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 21, 2007, 05:19:06 pm
As promised I've updated Modular Tables in Wiki (http://www.hard-light.net/wiki/index.php/Modular_Tables).

Please check my English and knowledge  :p.
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 21, 2007, 07:04:44 pm
EDIT: Note that, due to short-circuiting, this only causes problems with modular star tables. Since few people ever touch stars.tbl at all (much less make a tbm of it), it's easy to see how the bug stayed hidden.
I've used a stars tbm since the code was originally added, so most of the bugs have been caught by me anyway.  Something is funky with that code though, it was supposed to do something else at one point.  That doesn't appear to have hit CVS though, for some reason, so I need to go back and fix it.  :sigh:

But if you look at the parsing there in it's entirety, you might notice that is is totally broken at it's most basic level. ;)
Title: Re: Several questions about tbm and tbl
Post by: Turey on May 21, 2007, 08:39:07 pm
But if you look at the parsing there in it's entirety, you might notice that is is totally broken at it's most basic level. ;)

What, the fact that it keeps asking for "#End" when there's never a corresponding start? Or something more?
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 21, 2007, 10:36:05 pm
But if you look at the parsing there in it's entirety, you might notice that is is totally broken at it's most basic level. ;)
What, the fact that it keeps asking for "#End" when there's never a corresponding start? Or something more?
It looks for everything in a very specific order.  You can very easily end up with a tbm that doesn't work at all, or that completely skips sections if you only include latter ones.

Basically, everything needs to be in any order at all, and all sections need to be valid even if missing.  I'll get that fixed up later tonight though.  Still, a rather stupid bug, and it's pretty bad of us to not have noticed until now, especially considering that at least 3 devs (including me) have been through that same code in the past year or so.  :rolleyes:
Title: Re: Several questions about tbm and tbl
Post by: Wanderer on May 22, 2007, 01:48:30 am
At least one thing in the wiki...

Quote from: Wiki
Within the same priority place, they are applied in reverse alphabetical order (so zzz-shp.tbm is applied before aaa-shp.tbm).

Quote from: taylor
EDIT:  And just to make sure it's understood: the files are priority sorted only, not sorted alphabetically (excluding the fubar from the code bug).  They are only sorted in the reverse order they are found.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 22, 2007, 03:12:41 am
At least one thing in the wiki...

Quote from: Wiki
Within the same priority place, they are applied in reverse alphabetical order (so zzz-shp.tbm is applied before aaa-shp.tbm).

Quote from: taylor
EDIT:  And just to make sure it's understood: the files are priority sorted only, not sorted alphabetically (excluding the fubar from the code bug).  They are only sorted in the reverse order they are found.

I'm pretty sure he means that they are NOT ONLY alphabetically ordered. Alphabetical order is only applied to sort files within the same folder or within the same vp. Main sorting is based in mod folder order. At least this is the way it seems to work.

Check my previous tests with aa-shp.tbm, zz-shp.tbm and the vp'ed prueba-shp.tbm:
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 22, 2007, 03:27:01 am
At least one thing in the wiki...

Quote from: Wiki
Within the same priority place, they are applied in reverse alphabetical order (so zzz-shp.tbm is applied before aaa-shp.tbm).

Quote from: taylor
EDIT:  And just to make sure it's understood: the files are priority sorted only, not sorted alphabetically (excluding the fubar from the code bug).  They are only sorted in the reverse order they are found.

I made a point about that in the post you quoted, but removed it when even I had trouble understanding it. ;)

Files are never sorted alphabetically except when you build a list, and even then it's optional.  In this particular case we sort in reverse order rather than alphabetical order, since you get to do one or the other, but not both.

Basically, the wiki entry is misleading.  While it's true that you will often see files in the same place sorted in reverse alphabetical order, it's not guaranteed.  They are only sorted in priority order, as they are found, which is often alphabetical order in a VP.  When those files are then reversed, they end up in reverse alphabetical order.  But files in a VP are only found alphabetically if they are added like that in the first place.  So in a VP, there is nothing to say that aaa-shp.tbm is actually going to come before zzz-shp.tbm, even if they are in the same folder.

Usually they should be in alphabetical order in the first place, which means that they will be applied in reverse alphabetical order.  But that isn't guaranteed.  It really just depends on your VP util as to whether they are going to be in alphabetical order or not.  If it just reads every file off the disk in a raw fashion then they are likely to not be alphabetical order.  The cfilearchiver utility in the source code does not do any sorting for instance.  This means that files will be added in the order in which they exist in the directory, usually in order of creation.

It's the same thing for files on the hard drive btw, and not in a VP, only in this case they are usually not in alphabetical order.  Exactly what order they will be in depends mostly on the filesystem, but it is pretty much never alphabetical.  More often than not they are sorted in the order that they are created.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 22, 2007, 07:49:27 am
It's the same thing for files on the hard drive btw, and not in a VP, only in this case they are usually not in alphabetical order.  Exactly what order they will be in depends mostly on the filesystem, but it is pretty much never alphabetical.  More often than not they are sorted in the order that they are created.

Ooops, I think you should guarantee that files are read in alphabetical order from hard disk, (despite the exceptions "-", "_" or any other strange symbols, like Spanish "ñ" or French "ç", or "intelligent" sorting within Windows XP with 10Whatever being after 2Whatever, could cause).

Example: take in mind that your media vp patches are supposed to be loaded overriding media vp just because their names (mp_whatever vs. mv_whatever). If FS2 depends on creation date, it could be REALLY tricky and unusable.

BTW, the loading order from inside a vp is not really important because you are never going to pack conflicting tables or files within the same archive, unless you are a very bad mod designer.

I will modify Wiki ASAP saying that alphabetical order is not guarantied inside a vp. BUT I'll keep it for hard disk readings. If you don't guarantee that this is going to be CHAOS.
Title: Re: Several questions about tbm and tbl
Post by: Wanderer on May 22, 2007, 09:30:12 am
ARSPR... i removed that line from the wiki already.. Lets first wait for taylor's (and/or other coders) opinions and comments before adding anything related to this issue to the wiki
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 22, 2007, 10:00:37 am
ARSPR... i removed that line from the wiki already.. Lets first wait for taylor's (and/or other coders) opinions and comments before adding anything related to this issue to the wiki
I agree with you, let's leave the geniuses work.  :yes:

Title: Re: Several questions about tbm and tbl
Post by: taylor on May 22, 2007, 10:54:16 am
Example: take in mind that your media vp patches are supposed to be loaded overriding media vp just because their names (mp_whatever vs. mv_whatever). If FS2 depends on creation date, it could be REALLY tricky and unusable.
VPs themselves are sorted alphabetically, so there is no issue there.  It's just the files under data/ that aren't sorted, and I see no reason to change that behavior.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 22, 2007, 11:43:24 am
Example: take in mind that your media vp patches are supposed to be loaded overriding media vp just because their names (mp_whatever vs. mv_whatever). If FS2 depends on creation date, it could be REALLY tricky and unusable.
VPs themselves are sorted alphabetically, so there is no issue there.  It's just the files under data/ that aren't sorted, and I see no reason to change that behavior.

Well I can see one, although by far, it's not so critical as vp sorting. In fact, only Vp sorting is THE CRITICAL one.

Between releases, Mediavps does usually have a lot of official or unofficial updates (new models, small fixes, or whatever; for example I've got 9 user-patches + 4 HTL models in addition to 3.6.8. zetas+alpha patches). The best way to do this process is packing each "update" in one Vp. In this way you control how these updates are applied and in which order. But, maybe people just copy the additions to mediavps\data folder.

As there can be a lot of "updates" which are not really designed with any of the others in mind, if some kind of cross-effect happens between tables it can be a PITA to track it down. And if you don't know how the tables, which you don't remember what they have, are being loaded, the problem gets worse. With the vp use, at least, you know how they are loaded, but it seems that with just file loading is nearly random or luck.

So, I think that, if it is not too much coding work, adding this alphabetical criterion to file loading could be a good idea. Although you could have always used vps method (the most hygienic one). And OTOH I see no harm in an alphabetical file loading, (apart from your coding homework  ;) ).
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 22, 2007, 01:42:03 pm
So, I think that, if it is not too much coding work, adding this alphabetical criterion to file loading could be a good idea. Although you could have always used vps method (the most hygienic one). And OTOH I see no harm in an alphabetical file loading, (apart from your coding homework  ;) ).
You mean, aside from the massive performance decrease on game load and any time that the game does to disk for file loading?

There is actually a reason that individual files aren't sorted you know. :)
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 22, 2007, 02:41:22 pm
So, I think that, if it is not too much coding work, adding this alphabetical criterion to file loading could be a good idea. Although you could have always used vps method (the most hygienic one). And OTOH I see no harm in an alphabetical file loading, (apart from your coding homework  ;) ).
You mean, aside from the massive performance decrease on game load and any time that the game does to disk for file loading?

There is actually a reason that individual files aren't sorted you know. :)

OK then  :yes:

Unless you, the expert guys, have a different opinion, tomorrow, I'll update Modular tables section in Wiki posting a warning about putting conflictive tbms in the same folder or vp. It will say that there's no easy way to predict which one will be really applied.

(I think earth people, like me, shouldn't be messed up with technical info saying that the loading order will depend on how the vp was built or how your computer OS filesystem works ...)
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 23, 2007, 11:43:01 am
Modular Tables in Wiki (http://www.hard-light.net/wiki/index.php/Modular_Tables) is re-updated.

Please check my English and knowledge, again.

(Also, when -str.tbm is fixed, please post here so I re-test again).
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 23, 2007, 11:55:18 am
(Also, when -str.tbm is fixed, please post here so I re-test again).
It is fixed already, but CVS is down so I can't commit the changes.  If you want to go ahead and test it out then let me know and I'll post an updated Xt build later today.  It's not going to be much different than 0520 though.


EDIT: Oh, and someone probably needs to update the fireballs.tbl entry too.  After "$LOD:" there can be "$Light Color:", which takes 3 values in the 0 to 255 range.  This sets the color of light that is given off when that explosion is created.  And values set at "0 0 0" will not give off any light at all, just FYI.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 23, 2007, 11:59:46 am
Don't worry, I can wait till your next XT (for example with errors & warnings enabled   :p ). Don't compile a build just because of this issue.



EDIT -------------

I've decided to make further reorganization in the Modular Tables page. I think now it is clearer.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 23, 2007, 12:25:45 pm
EDIT: Oh, and someone probably needs to update the fireballs.tbl entry too.  After "$LOD:" there can be "$Light Color:", which takes 3 values in the 0 to 255 range.  This sets the color of light that is given off when that explosion is created.  And values set at "0 0 0" will not give off any light at all, just FYI.
I'll do but just several questions:

+ Is it a SCP addition? (Just to paint it in red).
+ Which are the default values if it is not specified?
+ RGB coded? GBR? BRG?
+ I suppose tbm supports it.
Title: Re: Several questions about tbm and tbl
Post by: taylor on May 23, 2007, 01:28:04 pm
+ Is it a SCP addition? (Just to paint it in red).
Yes, added after 3.6.9 was released (ie, in late January).  It will be an official feature starting with 3.6.10, in other words.

Quote
+ Which are the default values if it is not specified?
The default values are the ones that used to be hard-coded.  Those being:
255, 127, 31 for the LARGE1, LARGE2, MEDIUM, and ASTEROID explosions
191, 191, 255 for the standard WARP effect
191, 255, 191 for the KNOSSOS WARP effect

Quote
+ RGB coded? GBR? BRG?
RGB, to keep it the same as all other tbl color specifications.  There is no alpha component however, so it's only valid to have the 3 values for RGB.

And as stated earilier, values of 0, 0, 0 will give off no light at all when that particular effect occurs.  If you wanted dark/black light you would have to make at least one of the RGB values "1", or simply use 1, 1, 1 for a uniform dark color.

Quote
+ I suppose tbm supports it.
Yep.  And it's completely optional in both tbl and tbm.
Title: Re: Several questions about tbm and tbl
Post by: ARSPR on May 23, 2007, 01:33:18 pm
+ Is it a SCP addition? (Just to paint it in red).
Yes, added after 3.6.9 was released (ie, in late January).  It will be an official feature starting with 3.6.10, in other words.

I won't update Wiki yet then.

When 3.6.10 is out, at least in RC version, I'll post it.