Author Topic: Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"  (Read 661 times)

0 Members and 1 Guest are viewing this topic.

Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"
Hi all,

It seems that Knossos Explore tab is acting up on me and not displaying any mods.
Knossos log:
Code: [Select]
INFO:Thread-7:util.download:Downloading "https://cf.fsnebula.org/storage/repo.json"...
ERROR:Thread-7:tasks.work1: Failed to decode "#public"!
Traceback (most recent call last):
  File "C:\knossos\knossos\tasks.py", line 107, in work1
  File "C:\knossos\knossos\repo.py", line 128, in parse
  File "C:\Python36\lib\json\__init__.py", line 354, in loads
  File "C:\Python36\lib\json\decoder.py", line 339, in decode
  File "C:\Python36\lib\json\decoder.py", line 355, in raw_decode
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 1244785 (char 1244784)

For whatever reason, it is not recognising repo.json as a valid json

I was using Knossos for quite a while now but recently it broke down, could you guys help take a look?

Many thanks
« Last Edit: January 12, 2023, 11:30:04 pm by canemxiao »

 
Re: Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"
Ok... I played around with Knossos and I think I've found the issue.

The reason seems to be downloading of https://cf.fsnebula.org/storage/repo.json stopped midway leaving a partially formed mods.json under C:\Users\<username>\AppData\Roaming\knossos which Knossos failed to read.
I downloaded the JSON using a downloader and replaced mods.json with the downloaded file, which fixed the issue.

however, that partially formed JSON is already 116MB+ and I suspect that due to python being slow as all hell, it never finished downloading before the timeout and then the connection closed.
the full JSON is around 270MB which brings me to the questions:

1) why does have to be this big?
2) could we do some optimisation on the size or how it downloads mod list?

« Last Edit: January 13, 2023, 07:07:07 am by canemxiao »

 

Offline jg18

  • A very happy zod
  • 210
  • can do more than spellcheck
Re: Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"
Hi there! Thanks for finding this. The JSON file probably used to be a reasonable size but kept growing as the number of mods listed on Knossos grew.

There are a variety of options for a solution, such as providing only limited info about each mod and then providing details on demand, but the result may be chatty client-server I/O. Another option is versioning the JSON and sending delta updates from the server, rather than the full JSON file every time. Other solutions undoubtedly also exist.

Unfortunately I do not have bandwidth to look into this but perhaps someone else does. You may want to file a GitHub issue (the repo is ngld/old-knossos IIRC).

The workaround you found may be the best option for now.

EDIT: an even simpler code fix is to send zipped JSON and unzip client-side. The JSON should compress well, being so text heavy and likely with similar language throughout. I bet bzip2 would work well, but not sure offhand if Python supports that.

EDIT 2: .7z would be better than bzip2. Knossos already uses it for mod data.
« Last Edit: January 15, 2023, 01:15:12 am by jg18 »

 

Offline Erkhyan

  • 24
    • My ramblings
Re: Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"
I’ve recently gotten into FS2 again and immediately ran into that huge JSON file issue. FYI, I randomly checked, and a 7z-compressed version of the file would be roughly 10mb, versus how impractically large it is now.

 

Offline Cyborg17

  • 29
  • Life? Don't talk to me about life....
Re: Knossos 0.14.3 Explore tab not working "Failed to decode '#public'!"
We've had reports of this issue on discord already, and the stopgap to fix this issue until we get knossos to deal with it better is to delete %appdata%/roaming/knossos/mods.json and %appdata%/roaming/knossos/mods.json.etag

This forces knossos to retry the download. 

Taylor has been doing some work on knossos lately, but he's not currently able to fix this issue.


The cause for this issue is that the download sometimes fails without knossos realizing it.