Site Management > Site Support / Feedback

New forum search

(1/6) > >>

ngld:
There've been quite a few complaints about the existing forum search. So I thought I'd write my own as a fun side project.
You can see the result here: https://hlp-search.tproxy.de/

It's indexing every publicly available post. However, the indexing process hasn't finished (at the time of writing) so quite a few posts might be missing. The above link will let you know how far along the indexing process is.
The search engine itself is fairly straightforward. You enter you search query and get results. Advanced searches (i.e. find this -but -not -this, "Find this exact sentence.", show only results where +this word appears, etc.) are also possible. Here's the full documentation of the query syntax. The input field has autocomplete but that's just based on all of the indexed thread titles so it might lead to some interesting results. You'll also get suggestions like "Did you mean ...?" if you misspelled something.

Anyway, this was a fun project. Let me know if this is actually useful to anyone, if you're interested in the source code / how it works or want some kind of change.

EDIT: Mentioned issues were fixed. New posts are added to the index every 2 hours.

jr2:
Can we get a link to it in the top navbar?  Maybe Search > ngld improved search -OR- legacy search?

Goober5000:
That's jumping the gun; this should be thoroughly tested before we put links anywhere.  We don't want to overload ngld with tech support requests for this and Knossos. :)  Plus this is limited to publicly available posts, not the whole forum.

But this is cool!  Forum search has sucked for quite a long time; this is a welcome development.  If it proves to be substantially more accurate than the forum search, then I would be in favor of indexing the entire forum and making this the default.

Novachen:
A nice addition, sure :)

I personally had never issues with the forum search, however. Was able to find everything so far without problems :).

I think the most problem peoples in forums have, that they simple search wrong :D

ngld:
From now on the indexer should add new posts every 2 hours which should allow you to find most recent posts (can do this more frequently but I don't want to put unneccessary load on the HLP server).

I've noticed that it's kind of slow if noone's used the search in a while (results take ~6 seconds in that case) but quickly gets faster with every following search request (something like ~3s, ~1.5s, ~0.9s). I haven't looked into why this happens but I guess the search server (Elasticsearch) empties it's memory cache when it sits idle for too long. Either that or the memory gets swapped out. I might be able to fix that but I'm not sure if anyone's actually interested in that. So far, I've seen less than 1 search per day.

It's possible to add additional filters (i.e. show only results from these threads / forum /...) and fairly simple. Indexing private stuff is more complicated because then the search frontend somehow needs to know which forums the user can access and restrict the search results to those forums. Most of the complicated stuff here is somehow retrieving that list (either from the forum DB or through a PHP script).

Finally, it'd be possible to make the default search fields in the forum use my search but as Goober said, I'd rather make sure it works as expected first. I could probably add a link the my search engine which allows you to run the same search using the forum's search. Something like "retry with the old forum search". Though that'd only help if we made my search engine the new default. We could probably make this a profile setting or something like that... then everyone can decide for themselves which engine they want to use by default.

Navigation

[0] Message Index

[#] Next page

Go to full version