Nah, I don't think that's right.
I agree that colour perception is largely an internal process. However, your claim that because several different mixes of attributes can appear as the same colour, colours somehow don't exist in reality is false. It's merely demonstrative of the limitations of our visual perception - we can't make a distinction between the two things so our brain dubs them the same colour, but that doesn't mean it's somehow an incorrect distinction.
I can see the value in your argumentation, but I think our definition of colour is a bit different.
Let's try this:

Here we have three dithered fields. Each field consists of blocks of four pixels, enlarged in the upper left corner for each picture.
These appear as the same grey colour. Why? Because on each image we have a situation where every other pixel has two sub-pixels lit up, and every other pixel only one.
On our display hardware, every square should render identically as far as the light intensity goes, and it should appear the same level of grey.
Now, our ability to see colours on computer screen hinges on dithering; our inability to distinguish sub-pixels as individual entities makes it possible to mix red, green and blue in varying quantities to produce a more or less wide gamut of colours.
However, the fact that our rendering displays render all three images similarly doesn't mean they are the same. Their individual properties are different, and each is made of completely different colours pixel-wise, as zooming in reveals.
Now if I were to compare the display to our visual perception, one could say that both arguments have value: All objects are grey in our display, but they don't have grey in reality - hence the greyness is an illusion caused by our display hardware.
The other argument says that each of the objects still have individual colours. One made of cyan and red, one made of green and magenta, and one made of yellow and blue. The fact that display renders them similarly doesn't mean the images themselves don't have these properties.
I don't know if this portrays what I'm trying to say: I'm saying that colour as a physical property does exist in various ways and we can identify them, but limitations in our perception sometimes make us identify various physical properties as the same colour.
You're saying that colour as a concept exists just inside our heads. In my opinion, this undermines the abilities of human visual perception quite a bit, because even though it is limited, it still can distinguish between quite a bit of clearly differently coloured objects.
You could say that my concept of colour is primarily involved in the physical side of things rather than psychological or neurological perception. Mixtures of properties aside, we can still identify spectral colours correctly within the visible spectrum...
TL;DR:
You say because we identify various properties as same colours, colours don't exist.
I say there are more colours than we can identify, and thus some identify as the same.