"Indexed color mode" is where the image has a palette index (a set of numbers paired with a set of RGB color definitions) and then each pixel simply refers to the index number.
By tradition, the index value is one byte, which means that you are limited to 2^8 entries which is 256.
In order to have more color swatches, you want to use RGB mode, load the swatches, and then use only those swatches for the design, avoiding any blending, blurring, feathering, opacity, overlay, etc. (basically anything that will blend two of the swatches will result in a color that is not in the swatch list.
If you use 256 or fewer swatches in the design, you can then reduce it to indexed mode using either "exact" palette or making a custom palette with only the swatches you meant to choose and then setting it to that palette. In the second case, you probably can use dither etc when converting since you are forcing the palette to a known set of thread colors. And in the second case where you force the palette, and use dithering, you can use some blending etc.
I would recommend you test out the whole process on a few known designs before you spend {x} hours on a design that cannot be converted.
