First. Remember that colors can be arranged in a color wheel, but in reality that is not a circle, but a solid, that can come in different shapes, depending on what color model we want to choose.
Some initial Color solids
For example, the RGB-CMY solid is a cube.
Here is an HBS solid. As it is a 3D object we need to choose our parameters of "inverse". You can choose an inverse color on the same "slice" (A) or across the height of the solid (B) Depending on the model an inverse color of any of the top slice will be black in this case (C).
On this HSB figure (that can be either a cylinder or a cone) to find the opposite hue you need to add 180°, because it is using radial coordinates.
If you want, you simply choose an opposite color adding 180° to the choosen color.
We also can choose opposite colors seeing the RGB components as graphs, in this case, the range is from 0 to 255.
We add colors modifying this values, adding more color, sliding the graph upwards (or simply adding a bigger number)
You can see that if you do not assign 255 to any RGB slider, you have a gap.
This gap is the value of the opposite color:
Let us see how this reacts to some known opposite colors.
This makes changes to the RGB solid in 3 dimensions.
So the inverse of a light color is a dark color:
But if you want to stick to changes in the same plane, the graph itself provides the lower and the upper limit of the changes. Now the opposite of a light color is the opposite hue, but the same brightness:
So, at the end, it is simply a matter of counting your RGB values and make a series of subtractions.
Here is a related explanation to modify the same color to a lighter and darker one using the same methodology: How to make a given color a bit darker or lighter?
P.S. This is actually how different blending modes on different programs work. Making some simple arithmetic operations on the values of the colors. Inverse, Multiply, difference, etc.
Adding more information.
Other color modes
There are some other color modes that actually have different HUE opposites, for example, the LAB model has Green as opposed to Red, instead of cyan, and Yellow as opposite to Blue.