Installing it would edit the registry on your system, and FireFox would expect to use those registry entries on any system that you tried to use it on. FireFox Portable (and other Portable Apps) are optimized for use on a removable flash drive. See
What is a Portable App? and the following:
Privacy: Does Firefox Portable leave behind any personal data?
A. No. Firefox Portable has its disk cache disabled. It uses RAM for cache. All your cookies and other data is self-contained on your portable device. At no time is any personal data stored on the local machine, so even if someone tried to find it with an undelete utility, there would be nothing for them to find.
This would not be the case if you installed Firefox onto your flash drive, and if you did that, your flash drive FireFox might conflict with your version of Firefox installed on the Hard Disk, possibly. (If it tried to store settings, history, plugins, etc in the Hard Disk instead of the flash drive... esp if they are different versions.)