Eyes and screens are nonlinear in a complex way, so no simple difference calculations with RGB numbers are useful. Serious scientific research has been done and we have got some recommendations how RGB numbers should be used when one wants to estimate the difference between 2 colors. This is, of course, for deciding if texts are readable on usual sRGB displays.
One well known calculation formula is recommended by WWW Consortium. You can find it here:
I give a couple of screenshots of it.
The subjective distance between 2 colors is defined as contrast ratio. It's defined with the following formula:
Relative luminances of the colors are calculated from their RGB numbers with the following formula:
The formulas can seem complex, but actually they are elementary math, no university level math skills are needed. The same formulas are also implemented as online calculators for those who want the result without calculations.
Contrast ratio isn't "distance" in the same sense as geometric distance. Minimum value of contrast ratio is 1 (=no contrast at all), minimum geometric distance =0. If you need zero as the minimum, take a logarithm.
There's also another important difference. W3C's Contrast ratio is designed to help to keep webpages readable also when one cannot see well hue differences, so it's essentially black&white contrast only.
If you want to present as well the differences of luminances as differences of other color qualities, the problem is different. Many attempts are done to create plausible full color difference measures. There's a Wikipedia article of them:
I pick out one relatively easy to understand version from there. It's for colors presented in CIELab color system (=Lab mode in Photoshop) and the formula is known as CIE76 color difference.
Let your 2 colors have Lab color numbers L1,a1,b1 and L2,a2,b2. Calculate euclidean distance between the number triplets:
Conversion between RGB and Lab is tricky. You must learn quite thoroughly the meaning of Lab numbers to understand it. To understand Lab fully one must at first understand XYZ color system which is the basic model for seen colors. Full conversion formulas are beyond the scope of this answer, but you can find them with web searches. Here's one place:
You can also pick the needed formulas from Wikipedia articles. Start from here: https://en.wikipedia.org/wiki/CIELAB_color_space
My own opinions: I do not like CIE76 distance. It shows substantial distance between two blacks - two Lab colors with different a and b, but zero L. I would multiply CIE76 distance with a number which is zero if the more luminous color of the two colors under comparison is black. A proper multiplier could be the sum of the L numbers. Or some it's power with exponent < 1.
Proper scientists obviously have noticed the same, but their corrected formulas are more complex. The same Wikipedia article of color differences shows several of them.