8

What is the internal precision used by Adobe Illustrator? Or put another way, what internal representation does Illustrator use to store coordinate data?

Try this (I'm using CS6):

  • Zoom in as close as Illustrator allows (6400% in CS6).
  • Draw a circle.
  • Resize the circle to 0.001 × 0.001 pt.
  • Resize the circle to 10 × 10 pt.

Here's what I get when I do that:

Circle becomes square

This is obviously an extreme example, but I have experienced similar distortions in more realistic scenarios when objects have gone through extensive editing. To avoid this, I have had to resort in some cases to working at an enlarged scale and only resizing my artwork to final size at the very end.

Clearly, Illustrator's internal precision is not infinite. It appears that coordinates are ultimately rounded to an integer at some small scale (I assume some fraction of a point), such that rounding errors can accumulate, eventually culminating in visible distortion.

Can anyone shed some light on this?

Improve this question
6
  • @joojaa What version of Illustrator are you using? Maybe they improved this post-CS6?
    – SWB
    Dec 22, 2016 at 7:48
  • @joojaa You don't see this in CS5? That's interesting. It's definitely consistently reproducible in my copy of Illustrator CS6 (16.0.0 x64). Surely it's not a regression in CS6...
    – SWB
    Dec 22, 2016 at 7:58
  • By the way we could measure the precision by scripting. But im afraid that the way tour comments ahve scoped this question makes it offtopic on this site.
    – joojaa
    Dec 22, 2016 at 8:07
  • Ok i after tracking a CS6 I can finally can replicate this problem on CS6
    – joojaa
    Dec 22, 2016 at 16:01
  • I tried this with CS5 and ended up with an even stranger shape!
    – Ryan
    Dec 23, 2016 at 0:36

1 Answer 1

Reset to default
1

No computer application has infinite precision. Even CAD applications do not have much larger accuracy scale. They do however allow you to change the working unit, which changes your zero point location, whereas illustrator allways calculates everything in points. You should probably want to read what every computer scientist needs to know about floating point numbers, which by now applies to every computer users.

Theres nothing wrong in drawing at scale, that is essentially what the CAD applications do they only have a facility to tell the user this fact. See the application internally only stores numbers interpretation of those numbers is up to the frontend.

Anyway I can not replicate your problem:

enter image description here

Image 1: a 0.001 pt circle progressively scaled by x10 (image form CC2017 but results in cs 5 are comparable)

I can however replicate the problem on CS6

After having tracked down a cs6 version i can verify that this indeed happens.

I can even roundtrip a full double float value for example:

in  0100000001110111011010111110101110000101000111101011100001010010 
out 0100000001110111011010111110110000000000000000000000000000000000

So it seems that the internal value of the system is certainly not a double float. Single float perhaps? It would make sense for a early version of a hardware acceleration code!

I have to test whether that is the case in CC2017 tough.

Improve this answer
6
  • I understand that. (I'm actually a developer and engineer.) Obviously the internal precision of Illustrator (or any software) is not "infinite." I'm trying to determine what the actual limit is in Illustrator. Illustrator does not even appear to be using floating point, since even single-precision floating point could easily preserve the circle in my example. Rather, it appears to be truncating to an integer at some level.
    – SWB
    Dec 22, 2016 at 6:51
  • @SWB It probably is using a fixed point arithmetic, since its essentiallyu a postscript engine. Ive speculated about this before, i think they are transitioning to floating point they may be in some mixed state. The real problem is how adobe chooses to operate the transformation matrices which makes no sense whatsoever. In essence if you ever interactwith 2 objects with different transforms all matrices reset to identity. This would blow up even rudimentary expectations of precision.
    – joojaa
    Dec 22, 2016 at 7:28
  • RE: fixed-point arithmetic. That is my guess as well. I'm hoping somebody who actually knows will chime in and let us know for sure. I've tried Googling and turned up empty.
    – SWB
    Dec 22, 2016 at 7:52
  • How did you do your test? That indeed looks like a conversion to single and back to double. (I get almost the identical result going through the same conversion in C#, differing only in that C# truncates the lower 32 bits, while your example rounds.) Still, that doesn't really explain what Illustrator is doing, since even a single would round-trip the control points of a circle 10->0.001->10 without a perceptible degradation. Curious that you're only seeing this in CS6. I assumed it was something more fundamental to Illustrator generally, but maybe it's a bug.
    – SWB
    Dec 23, 2016 at 1:28
  • 1
    I found this post made by Illustrator developer Teri Pettit in 2003 (version 10/11 timeframe): groups.google.com/d/msg/adobe.illustrator.macintosh/KSqIpfpQHdU/…. She states that Illustrator rounds values to 0.0001 pt on save, but uses single-precision floats while the file is open. This is after the change from the EPS-based native format to the PDF-based native format, which happened with version 9, so this may still still be true today. However, it still doesn't explain the observed behavior in CS6, unless it's just a bug.
    – SWB
    Dec 23, 2016 at 1:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.