After some fruitful discussions here and elsewhere I've set out to find such a color scheme myself.
This is all work in progress. I will update as I proceed, but remarks and comments are welcome.
Description of methods
Comparing two sRGB colors
To compare two colors on my screen I need to compute a distance measure.
I have the colors as sRGB
values.
I am first converting them into CIE XYZ
values using the sRGB->CIE XYZ equations.
Then I convert them further then into CIE L*a*b*
values using the CIE XYZ -> CIE Lab equations.
In this format, I can compute the CIEDE2000 distance value of the two colors. The higher the value, the better distinguishable are the two colors.
Comparing two black-to-color gradients
I now want to compare two sRGB colors that each generate a black-to-color gradient.
This gradient is computed in nGrad
colors. Because too dark colors always will be indistinguishable, I define a darkness cutoff below I do no longer care. For this darkness cutoff I convert the sRGB
values into HSV
values and require a minimum V-value of Vmin
.
When I now compare two colors that each generate a gradient, I compute the CIEDE2000
distance from one color of gradient#1 to all other colors of gradient#2 which are above Vmin
and note the smallest of these values, i.e. the least distinguishable pairing. I repeat this for the other colors of the gradient that are above Vmin
. The overall distance between the two gradients is then defined by the minimum value of the individual values.
In essence, I define the distance between two gradients as the minimum-distance of any color-pairing for colors above Vmin
.
Sorting and filtering sets of black-to-color gradients
With the above, I have a way to measure color distances between gradients. In a color set I have nCol
colors (each generating a gradient), so I compute the distances between all of them. When I say "compute the distance between two colors" I really mean I compute the distance between their respective gradients as described before.
I now sort the list of colors such that the first color is the one with the largest distance to its closest neighbor color. i.e. each color computes the distance to all other colors, and the smallest distance of these measurements is used for the sorting.
Next, I iteratively add the remaining colors, always picking the one which has the largest minimum-distance to the colors already chosen. In essence, I always pick the color which is the most distinguishable from the already chosen one.
On top of that, I define a DeltaE2000 minimum
value. Any color which isn't at least that distant from its nearest partner gets completely removed.
Testing
As a starting point, I create a somewhat random test set of similar and dissimilar colors that generate gradients. Grey squares indicate colors that fall below my Vmin = 0.1
threshold and are ignored for evaluation:

Applying the method from above and using a filter threshold DeltaE2000min = 10
gives me the remaining, sorted set below. As I've rotated the image counter-clockwise for easier display, the sorting is from bottom to top:

The result is pretty much as expected, giving me some confidence in the results.
I can now apply the same methods to the Light-24 color set from my original posting.
Original:

After sorting and filtering:

Note, that only two colors where dropped but that the sorting now allows me to pick N<=22 colors with maximum discernibly (as defined above) by starting with the bottom most color and going upwards as required.
What's next?
As I'm somewhat confident in the methods now, the real 'search for the scheme' can begin. I haven't completely made up my mind, but I think I will start with a somewhat randomized approach where I iteratively pick a random bright color, test it against the set, and add it if sufficiently different by the defined metric.
When no more color can be found after some break-up criteria, I will sort-filter that list. But it was a long day with lots of new stuff learnt, so this project is a bit postponed for now.
Update
I finally came around finishing this. I did two approaches: One starting with 0 colors and just randomly adding as described above, and two starting with a reasonably 'spread out' color scheme and just adding colors there.
In either case, not surprisingly too harsh restrictions based on "considered dark shades" or "minimum color difference) limit the number of successfully added colors dramatically. In the end, I settled on accepting colors with HSV V >= 0.2
and a CIEDE2000 distance of >= 5
. With theses settings, palettes with ~30 colors seem doable.
Below are the three "best" examples I came up so far. One based on the Light-24 scheme from above, the other based on a palette starting out with RGB fractions, the final one totally random. The schemes are again sorted such that the color distance between columns is biggest on the left and decreases to the right.
39 colors based on the Light-24 scheme
RGB values as CSV
39 colors based on the Light-24 scheme
NAME, R value, G value, B value, minimum distance to all previous colors (gradient)
Color # 1 , 000 , 181 , 247 , 99.99
Color # 2 , 246 , 249 , 038 , 43.43
Color # 3 , 255 , 000 , 146 , 40.61
Color # 4 , 034 , 255 , 167 , 26.28
Color # 5 , 106 , 118 , 252 , 22.47
Color # 6 , 228 , 143 , 114 , 21.33
Color # 7 , 247 , 255 , 228 , 15.79
Color # 8 , 214 , 038 , 255 , 13.97
Color # 9 , 078 , 255 , 000 , 13.39
Color #10 , 255 , 202 , 255 , 12.81
Color #11 , 013 , 249 , 255 , 12.13
Color #12 , 255 , 080 , 091 , 9.79
Color #13 , 182 , 142 , 000 , 8.95
Color #14 , 255 , 150 , 022 , 8.56
Color #15 , 232 , 255 , 121 , 8.09
Color #16 , 110 , 137 , 156 , 8.05
Color #17 , 255 , 109 , 224 , 7.87
Color #18 , 254 , 212 , 196 , 7.23
Color #19 , 188 , 113 , 150 , 6.67
Color #20 , 255 , 205 , 126 , 6.53
Color #21 , 254 , 000 , 206 , 6.33
Color #22 , 128 , 204 , 000 , 6.31
Color #23 , 000 , 255 , 217 , 6.19
Color #24 , 210 , 255 , 192 , 6.17
Color #25 , 168 , 255 , 109 , 6.16
Color #26 , 201 , 251 , 229 , 6.03
Color #27 , 220 , 088 , 125 , 5.93
Color #28 , 000 , 254 , 053 , 5.60
Color #29 , 149 , 240 , 255 , 5.59
Color #30 , 071 , 155 , 085 , 5.56
Color #31 , 255 , 098 , 061 , 5.52
Color #32 , 146 , 255 , 234 , 5.45
Color #33 , 255 , 171 , 187 , 5.28
Color #34 , 126 , 125 , 205 , 5.11
Color #35 , 238 , 166 , 251 , 5.09
Color #36 , 227 , 238 , 158 , 5.05
Color #37 , 255 , 234 , 149 , 5.03
Color #38 , 255 , 232 , 255 , 5.02
Color #39 , 138 , 255 , 197 , 5.01

38 colors based on a scheme using RGB fractions.
38 colors based on a scheme using RGB fractions.
NAME, R value, G value, B value, minimum distance to all previous colors (gradient)
Color # 1 , 128 , 128 , 255 , 99.99
Color # 2 , 000 , 255 , 000 , 52.03
Color # 3 , 255 , 128 , 000 , 31.31
Color # 4 , 000 , 250 , 255 , 24.75
Color # 5 , 255 , 000 , 128 , 23.13
Color # 6 , 255 , 255 , 000 , 18.63
Color # 7 , 255 , 255 , 244 , 15.51
Color # 8 , 255 , 000 , 255 , 14.15
Color # 9 , 084 , 168 , 252 , 12.61
Color #10 , 255 , 000 , 000 , 12.56
Color #11 , 202 , 255 , 156 , 11.50
Color #12 , 128 , 000 , 255 , 10.34
Color #13 , 255 , 143 , 187 , 10.13
Color #14 , 081 , 255 , 194 , 8.73
Color #15 , 255 , 186 , 171 , 8.50
Color #16 , 255 , 206 , 137 , 8.23
Color #17 , 255 , 128 , 255 , 7.99
Color #18 , 174 , 255 , 246 , 7.30
Color #19 , 134 , 255 , 000 , 7.27
Color #20 , 000 , 000 , 255 , 6.93
Color #21 , 255 , 255 , 128 , 6.90
Color #22 , 000 , 128 , 255 , 6.77
Color #23 , 255 , 190 , 255 , 6.67
Color #24 , 000 , 255 , 104 , 6.61
Color #25 , 255 , 167 , 000 , 6.51
Color #26 , 255 , 230 , 255 , 6.46
Color #27 , 255 , 128 , 128 , 6.13
Color #28 , 255 , 142 , 096 , 5.87
Color #29 , 255 , 206 , 000 , 5.65
Color #30 , 204 , 255 , 000 , 5.56
Color #31 , 252 , 084 , 168 , 5.56
Color #32 , 168 , 084 , 252 , 5.53
Color #33 , 255 , 085 , 046 , 5.46
Color #34 , 255 , 252 , 195 , 5.23
Color #35 , 128 , 255 , 128 , 5.19
Color #36 , 204 , 255 , 209 , 5.17
Color #37 , 216 , 229 , 255 , 5.11
Color #38 , 252 , 168 , 084 , 5.09
32 colors created by pure random choice
RGB values as CSV
32 colors created by pure random choice
NAME, R value, G value, B value, minimum distance to all previous colors (gradient)
Color # 1 , 255 , 255 , 000 , 99.99
Color # 2 , 255 , 113 , 255 , 47.77
Color # 3 , 000 , 255 , 255 , 31.95
Color # 4 , 255 , 072 , 026 , 23.92
Color # 5 , 000 , 255 , 069 , 20.28
Color # 6 , 251 , 245 , 255 , 17.57
Color # 7 , 255 , 150 , 000 , 14.69
Color # 8 , 199 , 255 , 146 , 10.95
Color # 9 , 255 , 137 , 176 , 10.68
Color #10 , 255 , 253 , 205 , 8.20
Color #11 , 164 , 255 , 000 , 8.10
Color #12 , 177 , 255 , 255 , 7.69
Color #13 , 255 , 168 , 161 , 7.38
Color #14 , 000 , 255 , 201 , 7.37
Color #15 , 255 , 198 , 113 , 6.79
Color #16 , 255 , 082 , 112 , 6.05
Color #17 , 255 , 210 , 180 , 5.99
Color #18 , 255 , 114 , 038 , 5.97
Color #19 , 255 , 088 , 185 , 5.95
Color #20 , 255 , 172 , 255 , 5.72
Color #21 , 090 , 255 , 000 , 5.68
Color #22 , 156 , 255 , 172 , 5.64
Color #23 , 202 , 255 , 217 , 5.56
Color #24 , 255 , 169 , 109 , 5.51
Color #25 , 184 , 225 , 255 , 5.50
Color #26 , 255 , 239 , 137 , 5.46
Color #27 , 136 , 255 , 223 , 5.34
Color #28 , 138 , 255 , 108 , 5.27
Color #29 , 255 , 222 , 255 , 5.22
Color #30 , 000 , 255 , 131 , 5.21
Color #31 , 255 , 182 , 000 , 5.17
Color #32 , 223 , 255 , 106 , 5.10