Interesting question but, with some faulty assumptions.
The human eye is not equally sensitive to all colors.
The responsiveness of out eye starts from not perceiving a wavelength at all, below ultraviolet, starting to perceive blues, greens, reds, and the sensitivity starts do decay next to infrared.
Here is a fake graph.
But our eyes because the fisiology behind it can be fooled by using just some combination of narrow bands.
If the eyes work as you expected would look like this
Here I made a color chart based on HSB, this is Hue, Brightness, and Saturation.
If I extract the brightness channel, I have what I expected to be when I made this chart in the first palace. All colors in the now disappeared white circle have in fact at least one component #FF.
But as the human eye is neither mathematical or theoretical, we do not perceive as this.
If I want to emulate the perceived brightness, I need to simulate the responsiveness of the eye, and I get something like this.
Which have sense, in the center, we have photons firing from all 3 led colors, therefore it is brighter.
We are more sensitive to Green, followed by green, and last from blue. So Green (bright) and red (bright) together gives yellow perceived as the brighter angle of the wheel.
That is why there are a lot of different 3D models to represent color, RGB cubes, Color wheels, Color cones, Double cones, spheres, amorphic spheres as the Munsell bubble, etc.
And each of them has a different purpose.
There are different meanings here. All colors on the 4th image have the exact same Brightness value of the pixels. This brightness is given by the max value of the pixel. There are no subtle differences there.
But that is only because someone on that specific COLOR MODEL made it that way.
But that is not a Perceptual color model. A perceptual color model would be the Munsell "Buble", but as each human perceives things differently it cannot be a mathematical universal color model. Some people have different types of daltonism, for example. That is to illustrate that each of us could potentially see "brightness" differently. This can also apply to the viewing conditions.
Or eyes can also be "tired" of seeing one color, so this brightness varies inclusive in our own eyes.
I'll post an example of this.
Read carefully before scrolling down
Cover one eye with one hand now. Don't ask, just do it and keep reading!
Now stare at the red rectangle for some 20seconds. After that, scroll down fast and alternate viewing the ball's image with your left and right eye, in rapid succession and see the differences.
I now broke one of your eyes and de-calibrated it.
You can not rely on a perceptual model...
Original ball image