Human perception isn't the same for all colors. Our eyes have different color pigments which absorb different frequencies of light.
There's a bit about this over in Physics.SE: Why do green lasers appear brighter and stronger than red and blue lasers? From this question a chart is presented that shows the absorption of different frequencies of light. The three cones are represented by their colors and the rods are the dotted line.
Note here that that something that is in the blue range, while it will trigger the blue cones, but poorly trigger the rods or the green or red cones. Meanwhile, something that is deep red will trigger the red and a bit of the green cone, but not much of the rod or any of the blue cone. The sum of the absorbance of the three pigments, peaks in the green making green look the brightest.
There's even more about this in the cone sensitivity curve found at Visible Light and the Eye's Response and the spectral sensitivity wikipedia page. When looking at these graphs take care to note that many of them have been normalized into a 0 .. 100 scale.
There is also an aspect of distribution of the various color cells. More on this can be read about in "Blue" Cone Distinctions at HyperPhysics which notes that outside of the Forvea Centralis (most densely packed area of the retina for maximum sensitivity) they are only about 2% of the total count of cone cells (which then suggests that there is a 'blue amplifier' in our perception of the signal). The reasoning for this is likely because the blue light will get bent differently than red or green and we would have significant issues with chromatic aberration in our own eyes if we were to sample blue outside of the main focus significantly (just thinking about that gives me a headache). The root of the color vision tree at HyperPhysics is Color Vision Concepts which is a good read.
This predisposition for green being the important color can be seen in the various Bayer filters used in digital cameras which sample green more than red or blue.
Bryce Bayer's patent (U.S. Patent No. 3,971,065) in 1976 called the green photosensors luminance-sensitive elements and the red and blue ones chrominance-sensitive elements. He used twice as many green elements as red or blue to mimic the physiology of the human eye. The luminance perception of the human retina uses M and L cone cells combined, during daylight vision, which are most sensitive to green light.
From Wikipedia Bayer Filter: Explanation
This can again be seen being utilized in the program ppmtopgm which converts a color image to grayscale. It uses the formula:
l = .299 r + .587 g + .114 b - note that green is more than twice as the other colors combined when creating the gray value, and 5x more than blue. Btw, be sure to read the quote in that man page... its quite amusing - especially when taken in context of its origins)
And thus, why green appears to be brighter than the other colors.