I think you are asking about the graphical implementation of an overshoot, commonly used in typography. In short, overshoot is added to letters like A and O (pointy or round—like circle in your example) to visually make them look the same height as f.ex. H and X ("flat", rectangle-like).
The sources of the Wikipedia article suggest overshoot of 1–3 % or 5 % for O. Keep in mind these are suggestions for typefaces and differ from font to font. You're better of trying different ratios and choosing one that is the most æsthetically pleasing in your opinion. Of course you could use the given percents as a starting point.
In practice:
- Align your rectangle and circle by their centers
- Choose a ratio—say, 5 %
- Resize the circle by its center using double the value you've chosen (ie. 10 % in this example) as in typography the 5 % is added only vertically with the baseline as pivot point.