Do We See Colors in the Same Way?

This question is not new: when I see red, how do I know that you don't see blue and are just calling it "red"? I don't think this is question is impossible to answer, even though it seems that the only way to know would be to upload your mind into someone else's brain. You'll have to allow me one assumption, though: that the brain "invented" the concept of red in order to react instinctively and quickly to danger, thus allowing increased chances of survival, procreation, and domination of whatever gene mutation allowed the concept of visually perceiving red to form. Obviously, the brain didn't invent anything, especially not intentionally.

Thought Experiment

Way back, two groups of people split from each other. One stayed on continent A, and one moved to B. On A, a very nutritious fruit has a lot of 300 nm light given off, and a certain predator has fur that gives off 700 nm light. On continent B, it's the other way around. Each group's brains would begin to link the predator's coat color with the concept of red. If a member of A is then moved to B, they may both agree on a word to call A's predator's coat color, but in their brains, they see them as two different colors. Following this out, the member of B may not last long on A due to their brain's interpretation of the predator's color since it will look blue and, therefore, not tied to the instinctual flight response.

Testing the Hypothesis

Determining if there are different responses to different colors, like a flight response upon seeing red, would be the first step. If there are, you'd then need to determine if the different responses are caused by different cone activations. For example, colorblindness could lead to a different response, but it wouldn't prove that they see red differently in their brains.

If or once a real difference in response is established between colors, you would then sample different groups of people, or even populations of animals, to see if different groups react differently to the prompted colors.