Author Topic: An effort to replace the POF format  (Read 16210 times)

0 Members and 1 Guest are viewing this topic.

Re: An effort to replace the POF format
I would love if FSO was able to natively read Collada, it would make model creation way easier, 90% of my personal model bugs come from buggy conversions from DAE to pof... specially with hierarchies and sub object positions.

 

Offline DahBlount

  • Moderator
  • 29
  • Alpine ☆ Cancer Tribulation
    • Minecraft
    • Skype
    • Steam
Re: An effort to replace the POF format
It will be more convenient for the modder but not much more convenient for the user unfortunately. DAE is an XML file type meaning it takes some time to parse however DAE can be stored as a C++ Object (PCS2 uses this, known as Collada DOM).

The question is, do we sacrifice faster parse times for more convenient import and extra goodies?
<Axem> yet still more insightful than #hard-light

<Axem> jad2.23 will just be cat videos

<DahBlount> So
<DahBlount> JAD2.2 is like that
<Axem> maybe
<Axem> it can be whatever you like!
<DahBlount> A Chocolate Sundae?
<Axem> sure

My models: GTF Gilgamesh - GTD Nuadha [Redesigning] - Ningirama [WIP] - GTG Zephyrus

 

Offline Iss Mneur

  • 210
  • TODO:
Re: An effort to replace the POF format
We already build binary cachefiles for POFs.  Why would we not do that for a DAE model?
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline DahBlount

  • Moderator
  • 29
  • Alpine ☆ Cancer Tribulation
    • Minecraft
    • Skype
    • Steam
Re: An effort to replace the POF format
Never said we wouldn't do that. The problem itself lies in the fact that XML files take longer to parse.

The main reason we're adding DAE to FSO is the animations and other user data bits that can be stored.
<Axem> yet still more insightful than #hard-light

<Axem> jad2.23 will just be cat videos

<DahBlount> So
<DahBlount> JAD2.2 is like that
<Axem> maybe
<Axem> it can be whatever you like!
<DahBlount> A Chocolate Sundae?
<Axem> sure

My models: GTF Gilgamesh - GTD Nuadha [Redesigning] - Ningirama [WIP] - GTG Zephyrus

 

Offline Iss Mneur

  • 210
  • TODO:
Re: An effort to replace the POF format
Okay, but how does a one time increase in the parse time, which is felt by the modder because the cachefiles are mostly portable, going to inconvenience the users?

The animations and other data would be stored in the cache as well, otherwise what would be the point of the caching?
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline DahBlount

  • Moderator
  • 29
  • Alpine ☆ Cancer Tribulation
    • Minecraft
    • Skype
    • Steam
Re: An effort to replace the POF format
I thought FSO parsed the .pof file still, albeit a more confined portion of it, alongside the cache file?
<Axem> yet still more insightful than #hard-light

<Axem> jad2.23 will just be cat videos

<DahBlount> So
<DahBlount> JAD2.2 is like that
<Axem> maybe
<Axem> it can be whatever you like!
<DahBlount> A Chocolate Sundae?
<Axem> sure

My models: GTF Gilgamesh - GTD Nuadha [Redesigning] - Ningirama [WIP] - GTG Zephyrus

 

Offline zookeeper

  • *knock knock* Who's there? Poe. Poe who?
  • 210
Re: An effort to replace the POF format
I thought FSO parsed the .pof file still, albeit a more confined portion of it, alongside the cache file?

It does. I believe the only thing the cache files contain are the BSP trees (which are information that the model file does not contain) and the model file as a whole always gets read and parsed by the game.

 

Offline Bobboau

  • Just a MODern kinda guy
    Just MODerately cool
    And MODest too
  • 213
Re: An effort to replace the POF format
index buffers
Bobboau, bringing you products that work... in theory
learn to use PCS
creator of the ProXimus Procedural Texture and Effect Generator
My latest build of PCS2, get it while it's hot!
PCS 2.0.3


DEUTERONOMY 22:11
Thou shalt not wear a garment of diverse sorts, [as] of woollen and linen together

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: An effort to replace the POF format
I thought FSO parsed the .pof file still, albeit a more confined portion of it, alongside the cache file?

It does. I believe the only thing the cache files contain are the BSP trees (which are information that the model file does not contain) and the model file as a whole always gets read and parsed by the game.

Actually, it's the other way around. The cache files are the geometry for rendering purposes. And I don't know how you come to the conclusion that the model files do not contain BSP data when the spec explicitly says it does :P
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Online Colonol Dekker

  • HLP is my mistress
  • 213
  • Aken Tigh Dekker- you've probably heard me
    • My old squad sub-domain
Re: An effort to replace the POF format
What's the sketch at the moment?

What are we leaning towards?

I look forward to seeing another ten years down the line when the engine is pushing out Advent Children quality animation. :yes:
Campaigns I've added my distinctiveness to-
- Blue Planet: Battle Captains
-Battle of Neptune
-Between the Ashes 2
-Blue planet: Age of Aquarius
-FOTG?
-Inferno R1
-Ribos: The aftermath / -Retreat from Deneb
-Sol: A History
-TBP EACW teaser
-Earth Brakiri war
-TBP Fortune Hunters (I think?)
-TBP Relic
-Trancsend (Possibly?)
-Uncharted Territory
-Vassagos Dirge
-War Machine
(Others lost to the mists of time and no discernible audit trail)

Your friendly Orestes tactical controller.

Secret bomb God.
That one time I got permabanned and got to read who was being bitxhy about me :p....
GO GO DEKKER RANGERSSSS!!!!!!!!!!!!!!!!!
President of the Scooby Doo Model Appreciation Society
The only good Zod is a dead Zod
NEWGROUNDS COMEDY GOLD, UPDATED DAILY
http://badges.steamprofile.com/profile/default/steam/76561198011784807.png

 

Offline zookeeper

  • *knock knock* Who's there? Poe. Poe who?
  • 210
Re: An effort to replace the POF format
It does. I believe the only thing the cache files contain are the BSP trees (which are information that the model file does not contain) and the model file as a whole always gets read and parsed by the game.

Actually, it's the other way around. The cache files are the geometry for rendering purposes. And I don't know how you come to the conclusion that the model files do not contain BSP data when the spec explicitly says it does :P

Yeah, you're right, I got things mixed up; as far as I can tell, the game generates some BSP data on-the-fly on modelload (model_collide_parse_bsp()), but of course the model file itself contains some, too.

So, yes, disregard what I said. :sigh:

 

Offline DahBlount

  • Moderator
  • 29
  • Alpine ☆ Cancer Tribulation
    • Minecraft
    • Skype
    • Steam
Re: An effort to replace the POF format
Okay. Even under the current system would DAE not take longer to parse even with the cache file?
<Axem> yet still more insightful than #hard-light

<Axem> jad2.23 will just be cat videos

<DahBlount> So
<DahBlount> JAD2.2 is like that
<Axem> maybe
<Axem> it can be whatever you like!
<DahBlount> A Chocolate Sundae?
<Axem> sure

My models: GTF Gilgamesh - GTD Nuadha [Redesigning] - Ningirama [WIP] - GTG Zephyrus

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: An effort to replace the POF format
So... surprisingly, no one has asked yet.

What kind of difference are we talking in parse length? Like 10% increase... 25%... 50%....? Cause, hell, if it takes a little longer to parse but we get all the goodies that come with it, would that not be worth it?

Seriously, how much of a time difference are we actually debating here?
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: An effort to replace the POF format
I am not sure that that discussion is actually relevant. Worst case, there'd be a lengthier cache generation step, i.e. a one-time increase in load times.

In any case, getting this feature to work right has to be the first priority.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: An effort to replace the POF format
Which was pretty much my point. :)
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline Bobboau

  • Just a MODern kinda guy
    Just MODerately cool
    And MODest too
  • 213
Re: An effort to replace the POF format
so it's not so much replacing POF with dae/collada, as it is coming up with a dialect of dae/collada that can hold all pof information and be converted to pof without any pcs rigging and giving the game engine conversion capability. Given how many mods distribute the index buffer cache files currently what you would basically be doing is circulating your source files along side the files the game is actually using, inflating the download size in the process.

I'm honestly surprized the IB cache is till being used, and it hasn't been either integrated into the pof or a generation algorithm using a hash map hasn't been made resulting in dynamic creation fast enough not to need a cache.
Bobboau, bringing you products that work... in theory
learn to use PCS
creator of the ProXimus Procedural Texture and Effect Generator
My latest build of PCS2, get it while it's hot!
PCS 2.0.3


DEUTERONOMY 22:11
Thou shalt not wear a garment of diverse sorts, [as] of woollen and linen together

 

Offline DahBlount

  • Moderator
  • 29
  • Alpine ☆ Cancer Tribulation
    • Minecraft
    • Skype
    • Steam
Re: An effort to replace the POF format
Would the SCP be opposed to using the pre-compiled VC++ 8/9/10 assimp.lib and dependencies to integrate Assimp or should I try to integrate the code itself? The lib works with MSVC 05, 08, and 10 but we would need to compile versions for MSVC 11 and 13 as well as the environments for other OSes.
<Axem> yet still more insightful than #hard-light

<Axem> jad2.23 will just be cat videos

<DahBlount> So
<DahBlount> JAD2.2 is like that
<Axem> maybe
<Axem> it can be whatever you like!
<DahBlount> A Chocolate Sundae?
<Axem> sure

My models: GTF Gilgamesh - GTD Nuadha [Redesigning] - Ningirama [WIP] - GTG Zephyrus

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: An effort to replace the POF format
Since we have to compile it on OSX and Linux anyway, I would recommend distributing the source files.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline Bryan See

  • Has anyone really been far as decided to use even go want to do look more like?
  • 210
  • Trying to redeem, but under Tiger Parents
    • Skype
    • Steam
    • Twitter
Re: An effort to replace the POF format
In 2014, Quanto, DahBlount and Vaktezraj mulled the effort to replace the POF format with a native OpenCOLLADA support. Their efforts have not materialize. So I decided to step in, and based on the latest version (2018). My GitHub is: https://github.com/bryansee/OpenCOLLADA. I've started coding it on my repository: https://github.com/bryansee/fs2open.github.com.

My original total-conversion project, as well as Wings of Dawn, demands such. So, the question is: where do I need to go from here?
Bryan See - My FreeSpace Wiki User Page (Talk, Contributions)

Full Projects:
Shattered Stars

Campaigns:
Lost in the Mist - Cyrene vs. Psamtik
FreeSpace: Reunited

Ships:
GTS Hygeia, GTT Argo, SC Raguel

Tools:
FSO TC/Game template

I've been under attack by Tiger Parents like Jennifer Pan...

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: An effort to replace the POF format
My original total-conversion project, as well as Wings of Dawn, demands such. So, the question is: where do I need to go from here?

Bryan, do not EVER presume to speak for other developers. If spoon needs OpenCollada support in FSO, he knows how to tell us (But since he's spoon and thus awesome and aware of the issues involved, knows that this isn't likely to happen soon).

Secondly, if you are serious about trying to implement this yourself, you need to do the following.
Step 1: Learn how pof reading works.
Step 2: Implement a collada loader.
Step 3: Find a way to transform the collada geometry into the internal formats the engine is using.
Step 4: Find a way to embed the metadata the pof format provides, either by embedding it in the .dae or by providing supplementary files.

This, by the way, will only get you to feature parity with the pof reading code, and that alone is, in my estimation, a multi-week project (as in, several weeks of concentrated coding effort). For a hobby project such as this, it's easily a multi-month endeavour, and that's assuming that the coder in question is familiar with the FSO code and actually knows how to write C++. This is NOT a beginner project, and despite your proclamation of being a "visionary modder and coder", your past attempts at making contributions to FSO do not fill me with confidence that you actually know what you're doing. I'd be happy to be proven wrong about this, of course.

EDIT:
Also, Bryan, you've been posting about your projects on this site for almost ten years now. In that time, you have completed almost none of them. Why should we have any confidence in your ability to finish this in a reasonable timeframe?
« Last Edit: February 16, 2018, 05:27:44 am by The E »
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns