I have to group the colors in as few groups as possible. For example, I would group gold and khaki under yellow.
In which group should I put lavender?
My color groups are the following:
I would list Lavender under the Blue group.
Even though I would if asked categorize Lavender under Blue like @Lucian it is worth noting that:
Colors can not really be categorized this way. For example the color Khaki depending on person, and exact composition of the Khaki, could be categorized under green. And Lavender could be under Red depending on what hue you think Lavender is (altough most would put that it is blue) There is a really good write up on this particular subject on XKCD blog under the header Color Survey Results
So the take after reading the survey summary should be. Be very careful with naming of colors. Not many people agree with you, and that nobody knows how to spell Fuchsia. There is not much point in trying to place the more varied names in a system as people do not agree with the hue let alone what super category that hue belongs to!