Author Topic: Special characters in non-retail features?  (Read 582 times)

0 Members and 1 Guest are viewing this topic.

Offline Novachen

  • 28
  • The one and only Capella supernova.
Special characters in non-retail features?
Heyya guys,

even i am more familiar with a complete own branch of builds (long story) this time i have to use the normal builds and encountered some problems with that.
One of the reasons i have to reregister in HLP after a whole decade after that bad things happenend back then (evil dispute with some coders, another long story).

in order to translate some user-content in a try to revive my native language-speaking community of the freespace games, i encountered some problems with scp-specific features which are present in some of the newer campaigns.

Things like the Fiction Viewer, subtitle-SEXP and user-created HUD Gauges does not seem to support Special characters like á or ö, even the HUD Gauge use the same font as every other HUD Element.
These special characters are not shown at all, even i use or create a font file that has this kind of characters included.
I tried to create a fonts.tbl with a Special Character Index for the Fiction Viewer, without success there. Even it would not help in the SEXP and HUD Gauges Problem.

How to change this with the given means? So without a recompile and the use of an complete own build, that was customized for the language?
The use of an own build is not the goal, because i dislike the idea completely that players have to use a specific build to play in a specific language.
Not to say, that this would remember me of the programming hell of early 2000s Games when some of them did have indeed own executables for every language.

Greetings,

Jenna
« Last Edit: November 20, 2016, 03:15:51 pm by Novachen »
Female FS2 pilot since 1999.

German Translations created by me:
Between the Ashes: Slaves of Chaos, FreeSpace Port, The Destiny of Peace, Silent Threat: Reborn, Awakenings (in development)

Upgrades and Bugfixes:
FSPort Mission Upgrade, Luyten Civil War, Into the Halls of Valhalla, VeniceMirror

Other projects:
Out of the Shadow: Nova Safiya Edition (Remake of the Out of the Shadow campaign, in development)



If you want to know, what my nickname means, -chen is a german diminutive term, so you can translate Novachen as something like Little Nova or Novalet.
Even my original meaning of this name is more like "Sweet pretty deadly (Super)Nova" ;).

 

Offline Yarn

  • 210
Re: Special characters in non-retail features?
I can help you, but just so I know what we're dealing with, what language are you translating to?
"Your fighter is running out of oil.  Please check under the hood and add more if necessary"
--strings.tbl, entry 177

"Freespace is very tired.  It is shutting down to get some rest."
--strings.tbl, entry 178

 

Offline Novachen

  • 28
  • The one and only Capella supernova.
Re: Special characters in non-retail features?
I can help you, but just so I know what we're dealing with, what language are you translating to?

Thanks for the reply Yarn :-)

Fortunately i am not translating into a complete different character set like Chinese or Russian  :drevil:

It is "only" german, so the special characters i am talking about are mainly Umlauts and a few accents, that are used in few words that were adopted from the french language. So proper Umlauts would be more important.
There exist a workaround in our language for the case, that you are unable to use these characters. The writing of 'ae' has the same meaning as 'ä'. But that would look very strange if i can use the proper characters in most of the game, but not in some menus. A uniform appearance would be better in this case.

Greetings,

Jenna
Female FS2 pilot since 1999.

German Translations created by me:
Between the Ashes: Slaves of Chaos, FreeSpace Port, The Destiny of Peace, Silent Threat: Reborn, Awakenings (in development)

Upgrades and Bugfixes:
FSPort Mission Upgrade, Luyten Civil War, Into the Halls of Valhalla, VeniceMirror

Other projects:
Out of the Shadow: Nova Safiya Edition (Remake of the Out of the Shadow campaign, in development)



If you want to know, what my nickname means, -chen is a german diminutive term, so you can translate Novachen as something like Little Nova or Novalet.
Even my original meaning of this name is more like "Sweet pretty deadly (Super)Nova" ;).

 

Offline Yarn

  • 210
Re: Special characters in non-retail features?
Ah, German. This shouldn't be too difficult, considering the game had a German release, and that I have all the data files from that version (except the voice clips and movies, which you probably don't need).

For proper German support, you need certain files from the German version of FreeSpace 2. Here they are:
https://www.dropbox.com/s/rkmw57hu6ia307o/fs2_german_data.zip?dl=1

You don't need the missions folder or pxohelp.txt, but you do need everything else. They include a font file containing various non-ASCII characters, a wider kills gauge, and some table files that contain German translations that aren't included with the English version of the game. In case you're wondering, here are all the supported characters:
Quote
all ASCII characters
è, ò, ù (lowercase only)
á, í, ó, ú (lowercase only)
é, É
â, î, ô, û (lowercase only)
ë, ï (lowercase only)
ä, ö, ü, Ä, Ö, Ü
ç (lowercase only)
ß (converted to ss)

If the font file from that pack is used as the first font, FSO should automatically switch its language to German; if it doesn't, or you're using a different font as the first font, then you will need to do it manually. If you run Windows, here's how to do it:
  • For FSO 3.7.4 and earlier:
    • Open the Windows registry (regedit) and go to the location where FSO stores its settings:
      • On 32-bit Windows: HKEY_LOCAL_MACHINE -> Software -> Volition -> Freespace2
      • On 64-bit Windows: HKEY_CURRENT_USER -> Software -> Classes -> VirtualStore -> MACHINE -> SOFTWARE -> Wow6432Node -> Volition -> Freespace2
    • Make sure there's a value in the list that's named Language. If there isn't, right-click an empty area, select "New", click "String Value", and name it Language.
    • Double-click the Language value and change its data to German. (The capital G might be important.)
  • For later versions:
    • Open the folder %APPDATA%\HardLightProductions\FreeSpaceOpen. You can type this into a Run prompt or an Explorer window.
    • Open fs2_open.ini.
    • In the [Default] section, look for a line that begins with Language=. If it exists, then change the text after the equals sign to German. (The capital G might be important.) If it doesn't, then add the line Language=German anywhere in that section.
    • Save the file.

Special Character Index has nothing to do with accented letters. Rather, it specifies where a handful of FreeSpace-specific characters (like the infinity symbol) are located in the font file. If you're using the font file that I've given you, you shouldn't have to touch this.

I believe the fiction viewer currently doesn't support non-ASCII characters; I will look into fixing this. Non-ASCII characters should work everywhere else, though, including in subtitles and in HUD tables. If, even after getting the game to run in German, you still can't get accented letters to work in those places, then send me a file where you tried unsuccessfully to use them so that I can figure out what's wrong.
« Last Edit: September 15, 2017, 12:15:58 am by Yarn »
"Your fighter is running out of oil.  Please check under the hood and add more if necessary"
--strings.tbl, entry 177

"Freespace is very tired.  It is shutting down to get some rest."
--strings.tbl, entry 178

 

Offline Novachen

  • 28
  • The one and only Capella supernova.
Re: Special characters in non-retail features?
Heyya Yarn, thanks for this reply.

but i am not sure if you get my problem.

I do not have problems to use these special characters in the game. I already use the german release of this game and i do not have problems to create translation of campaigns, that does not use any newer feature.
For example i translated near the complete FreeSpace: Port with Silent Threat: Reborn without a problem (not finished it yet, unfortunately), because this campaign does not use any special features, only the (t)strings.tbl. There i can translate it with the use of special characters that are showed ingame.

But there are not showed in user-created things like the Fiction Viewer, Starmap, Subtitle-SEXPs and HUD Gauges.

Actually i discovered the use of special characters leads to a crash of the game if they are used in the Subtitle-SEXP:
Code: [Select]
---------------------------
Error!
---------------------------
Token "Drìcke Feuer zum ìberspringen..."
      15
      25
      ( false )
      ( false )
      4000
      5000
   )
   ( key-reset-multiple "Spacebar" )
   ( hud-disable 1 )
   ( script-eval "scam_inact()" )
   ( script-eval "CutsceneKeyLock()" )
)
+Name: PreCutsceneStart
+Repeat Count: 1
+Interval: 1

$Formula: ( when
   ( not
      ( is-event-true-delay
         "PreCutsceneEnd"
         0
      )
   )
   ( set-camera-position
      -67908
      3645
      -71589
   )
   ( set-camera-facing
[...]
[ This info is in the clipboard so you can paste it somewhere now ]


Use Ok to break into Debugger, Cancel to exit.

---------------------------
OK   Abbrechen   
---------------------------

Actually it should be "Drücke Feuer zum überspringen..." (=Press fire to skip...).
So there is also a coding problem in the character set there. Interestingly it cause a crash and not a simple invisible or wrong character.


What i mean in the HUD Gauges is this from my current "Between the Ashes" translation:

You see, there are many Umlauts in the HUD, but the "Ladungen übrig" in the right hand side of the upper HUD misses it, it does not show the ü.

Can this be a problem in the behavior FSO uses its font file?
So that a text have to be marked as "german" to get the additional characters?

Because all the other text you see in this screenshot is stored in the (t)strings.tbl under the #german flag for multi-language (and safe mission updates) use.
But this HUD Gauge is only in the hud_gauges.tbl itself, aswell the text for the subtitle-sexp is in the .fs2 file.

So, there is a question, is it possible to create ids for this kind of text to translate it in a string file in FSO?

Because i think that this would work, then...

If not... that would need a build where this kind of things are possible, unfortunately.

So it seems, that i have the same problem if i use the expanded german font without any changes in retail with the english language setting which mess up the HUD completely.
Actually i thought that was fixed in FSO during the last couple of years?

Greetings,

Jenna
« Last Edit: November 21, 2016, 06:56:27 pm by Novachen »
Female FS2 pilot since 1999.

German Translations created by me:
Between the Ashes: Slaves of Chaos, FreeSpace Port, The Destiny of Peace, Silent Threat: Reborn, Awakenings (in development)

Upgrades and Bugfixes:
FSPort Mission Upgrade, Luyten Civil War, Into the Halls of Valhalla, VeniceMirror

Other projects:
Out of the Shadow: Nova Safiya Edition (Remake of the Out of the Shadow campaign, in development)



If you want to know, what my nickname means, -chen is a german diminutive term, so you can translate Novachen as something like Little Nova or Novalet.
Even my original meaning of this name is more like "Sweet pretty deadly (Super)Nova" ;).

 

Offline Yarn

  • 210
Re: Special characters in non-retail features?
You're encoding your files in UTF-8 (a Unicode encoding), which FSO does not currently support. You must save them in either ISO-8859-1 or Windows-1252. (ANSI will also work if it's really Windows-1252 on your system, which it probably is.)

The English and German fonts store their special characters (again, this does not mean accent letters) at different indices, and the game expects them to be located at a certain index: the one specified in the Special Character Index; or, if that field isn't present, then 164 if the language is German and French, and 127 if the language is anything else.
"Your fighter is running out of oil.  Please check under the hood and add more if necessary"
--strings.tbl, entry 177

"Freespace is very tired.  It is shutting down to get some rest."
--strings.tbl, entry 178

 

Offline Novachen

  • 28
  • The one and only Capella supernova.
Re: Special characters in non-retail features?
Thanks for this another reply, Yarn.

Indeed, Notepad++ and the Windows Editor seem to have the habit to save into UTF-8 . I have not paid attention to this  :banghead:

I was able to fix the HUD problem with it. It is now "Ladungen übrig" like intended as i saved the hud_gauges.tbl into ANSI.

However this does not fix the subtitle-SEXP error. Even i saved the .fs2 into ANSI it seems to have problems with a special character in it, that totally crash the mission.

Code: [Select]
Token "Sekund„rfeuer zum berspringen..."
      15
      25
      ( false )
      ( false )
      4000
      5000
   )
   ( key-reset-multiple "Spacebar" )
   ( hud-disable 1 )
   ( script-eval "scam_inact()" )
   ( script-eval "CutsceneKeyLock()" )
)
+Name: PreCutsceneStart
+Repeat Count: 1
+Interval: 1

$Formula: ( when
   ( not
      ( is-event-true-delay
         "PreCutsceneEnd"
         0
      )
   )
   ( set-camera-position
      -67908
      3645
      -71589
   )
   ( set-camera-facing
      -66543
      2997
      -70878
   )
   ( set-camera-position
      -68349
      3645
      -71026
      8000
      0
      1000
   )
)
+Name: PreCutscene2
+Repeat Count: 1
+Interval: 1
+Chained: 9

$Formula: ( when
   ( or
      ( is-event-true-delay
         "PreCutscene2"
         7
      )
      ( key-pressed "Spacebar" )
   )
   ( player-not-use-ai )
   ( unlock-primary-weapon "Alpha 1" )
   ( unlock-afterburner "Alpha 1" )
   ( reset-camera )
   ( set-object-position
      "Alpha 1"
      -66752
      2811
      -70374
   )
   ( set-object-position
      "Alpha 2"
      -67206
      2825
      -70979
   )
   ( set-object-position
      "Alpha 3"
      -67043
      2998
      -71057
   )
   ( set-object-facing
      "Alpha 1"
      -66749
      2811
      -70368
   )
   ( set-object-facing
      "Alpha 2"
      -67336
      2958
      -71235
   )
   ( set-object-facing
      "Alpha 3"
      -67336
      2958
      -71235
   )
   ( clear-subtitles )
   ( key-reset-multiple "Spacebar" )
   ( hud-disable 0 )
   ( set-player-throttle-speed
      "Alpha 1"
      100
   )
   ( script-eval "CutsceneKeyUnLock()" )
)
+Name: PreCutsceneEnd
+Repeat Count: 1
+Interval: 1

$Formula: ( when
   ( true )
   ( send-message
      "PVD Frontier"
      "High"
      "FRNT - CatchThem"
   )
   ( set-support-ship
      "Hyperspace"
      "<no anchor>"
      "Hyperspace"
      "<no anchor>"
      "<species support ship class>"
      0
      0

It should be "Sekundärfeuer zum überspringen..." (=Secondary fire to skip...)

Actually i am curious of why ANSI does not work in menus like the fiction viewer or the system map?


Any suggestions? :)

Greetings,

Jenna
« Last Edit: November 22, 2016, 06:45:41 am by Novachen »
Female FS2 pilot since 1999.

German Translations created by me:
Between the Ashes: Slaves of Chaos, FreeSpace Port, The Destiny of Peace, Silent Threat: Reborn, Awakenings (in development)

Upgrades and Bugfixes:
FSPort Mission Upgrade, Luyten Civil War, Into the Halls of Valhalla, VeniceMirror

Other projects:
Out of the Shadow: Nova Safiya Edition (Remake of the Out of the Shadow campaign, in development)



If you want to know, what my nickname means, -chen is a german diminutive term, so you can translate Novachen as something like Little Nova or Novalet.
Even my original meaning of this name is more like "Sweet pretty deadly (Super)Nova" ;).

 

Offline Yarn

  • 210
Re: Special characters in non-retail features?
That line has 33 characters, but I believe subtitles are limited to 31 characters per line. Try breaking it into two lines.

I don't know what "system map" you're referring to (I'm guessing it's a custom screen written in Lua and not the normal briefing screen), but I can explain why the fiction viewer isn't properly displaying non-ASCII characters. While the game's text files are encoded in Windows-1252, the font files use code page 437, which has a different set of characters at different indices (but at least the ASCII portion is the same in both). Because of this, code page conversion is required. Most text-based files that FreeSpace loads undergo this process, but fiction viewer and perhaps Lua files do not, thus causing problems with non-ASCII characters.
"Your fighter is running out of oil.  Please check under the hood and add more if necessary"
--strings.tbl, entry 177

"Freespace is very tired.  It is shutting down to get some rest."
--strings.tbl, entry 178

 

Offline Novachen

  • 28
  • The one and only Capella supernova.
Re: Special characters in non-retail features?
Okay Yarn,

yeah, cutting the line do the trick. But it have to be shorter than 31 lines. Interesting that i do not get an error that tells me about that.
Even if i edit this file in FRED2 there is no message that tells me, that this line is too long for the SEXP. Maybe i should not have switch to an english freespace years ago, then i maybe would know much more about all this :/.

Actually in the game source code there is a Sectormap that was planned to be used as a navigational map in the first Freespace. But it was never used by Volition, but the remnants can be used for own Maps in the game and it is in some user-campaigns. But no matter, without own scripts and text this feature does anything.

So there is currently no possibility to use special characters in such features at the moment?
Thats bad. Hope that will change in the future... thought that would be introduced since the TTF Support in the FSO branch :).

Maybe i have to do some digging into my own branch for that and to translate that code for FSO as a contribute for a public release. Because i have a Fiction Viewer that accept all characters i can use in the Windows Charmap. Damn thing.


So, then i think the problems that are solvable are solved. For the rest i have to find other solutions as it seems.

So thank you for the help, Yarn :)

Greetings,

Jenna
Female FS2 pilot since 1999.

German Translations created by me:
Between the Ashes: Slaves of Chaos, FreeSpace Port, The Destiny of Peace, Silent Threat: Reborn, Awakenings (in development)

Upgrades and Bugfixes:
FSPort Mission Upgrade, Luyten Civil War, Into the Halls of Valhalla, VeniceMirror

Other projects:
Out of the Shadow: Nova Safiya Edition (Remake of the Out of the Shadow campaign, in development)



If you want to know, what my nickname means, -chen is a german diminutive term, so you can translate Novachen as something like Little Nova or Novalet.
Even my original meaning of this name is more like "Sweet pretty deadly (Super)Nova" ;).

 

Offline Yarn

  • 210
Re: Special characters in non-retail features?
I think I found a way to have more than 31 characters in a subtitle line: make a message containing the desired text, then set the subtitle text to the name of that message.

Actually in the game source code there is a Sectormap that was planned to be used as a navigational map in the first Freespace. But it was never used by Volition, but the remnants can be used for own Maps in the game and it is in some user-campaigns. But no matter, without own scripts and text this feature does anything.
Can you give an example of a mod that uses this feature, as well as where I can edit its text?

So there is currently no possibility to use special characters in such features at the moment?
Not right now. I should be able to fix the fiction viewer, and if you point me to a mod that uses the starmap, I should be able to fix that too.

Thats bad. Hope that will change in the future... thought that would be introduced since the TTF Support in the FSO branch :).
While FSO's TrueType engine, NanoVG, does support Unicode, the game itself does not. I do intend to start an effort to implement Unicode support eventually; just don't expect it to happen soon.
"Your fighter is running out of oil.  Please check under the hood and add more if necessary"
--strings.tbl, entry 177

"Freespace is very tired.  It is shutting down to get some rest."
--strings.tbl, entry 178