If the Installer is incompatible Windows 10 will give an error message. And actually it should not to be anything to do with strange sounds.
But well, as far as i know, FreeSpace1 Retail can be installed and run without Problems on a 64-Bit System.
Run as Admin and Compatibility Mode to Windows 98 should do the trick. At least after you have updated the game to 1.06, not sure about the original and ST (1.03) version however.
But the Installer should even work them.
But well, it can be happen, that there is another process, that blocks FS.exe. For that look in "Analyze Wait Chain" for the FS.exe in the taskmgr windows task manager.