It Started Curved
The apostrophe first appeared in the printed universe in Italy, 16th century, as a curved shape to signify elision copied from handwritten classical Italian poetry. The apostrophe was equivalent to our "Gotchas" or "Wannas" in the sense that it was a way to take the stiffness of the text away by making it sound more human-like.
Here is an example of one of the first times the apostrophe was printed. It is from "Le cose volgari di messer Francesco" an Italian classic written by Petrarch, published by Aldus Manutius in Italy circa 1501.
Aldus Manutius supposedly copied this book from an original manuscript of Petrarch owned by a friend of his. He had a special typeface designed based on the handwriting of the poet, creating the slanted style we nowadays know as "italics." He had to introduce the apostrophe glyph in order to be able to copy the text exactly the way the poet had written it. Notice the apostrophe in the word(s) "l'honorata". Notice it had the curved shape.
It got Fashionable
This innovative "Italic" type and the colloquial stylish way of indicating elisions as a cute small mark gained immediate fame. Even when the typeface was protected by the Pope himself it started being illegally copied by counterfeiting artists. Copies of the book were printed in Lyon using a pirate copy of the typeface with apostrophe and all.
In 1501 Geoffroy Tory published Campoflori, a book that revolutionized French grammar and book publishing. It introduced, among other things, the apostrophe, copying the new fashionable Italian way of considering elisions stylish. This is an excert from the book. Notice the curved apostrophe.

In the same century and for the same reasons, the English language imported the apostrophe from French.
It got Straight
The apostrophe stopped being a novelty and became part of regular grammar. It kept its curved shape in printed texts. The straight apostrophe-like glyph, on the other hand, had different functions, such as denoting "prime". In handwritten documents, though, sometimes it was stylized as a straight tick (see Yorick's comment bellow).
With the invention of the typewriters and keyboards (1860s), the engineers were faced with the problem of fitting many characters (keys) in a small space. They came up with the idea of unifying similar looking characters in order to save space. Instead of having the left, right and straight "apostrophes" they decided to include only the straight one. Similar decisions were made about other characters common in typesetting such as the em dash and the en dash.
https://en.wikipedia.org/wiki/Typewriter#Typewriter_conventions
This reduced set of characters was codified in the world's first binary character set, the 5-bit Baudot code (1870). The set also included the straight double quotation as a single character. 7-bit ASCII inherited the same character set.
Since then all typed (and telegraphed) text was set using only the straight quotation marks. Curly ones were used only in traditional typeset text.
It got Ambiguous
It was only with the popularization of word processor systems that the curly apostrophes were reintroduced into the digital world. Systems like Word offered the feature to automatically substitute the straight apostrophe with curly quotation marks based on the context. It was the user's option to do so, though, and not all word processors had the feature. To add to the confusion, the standard keyboards did not include the curly quotation marks, so it was very hard for the regular user to type them intentionally.
When Unicode was created, one of the design principles was to "unify" characters. Characters from different scripts or languages that looked similar were to be unified and simplified as a single one. You would think they would have gone back to the origin and get rid of the straight apostrophe. Now, by this point, the confusion and liberal use of the straight and curly apostrophes and quotation marks was so big that Unicode decided to include both.
The Apostrophe Today
So, as it is today, both (the curly and the straight apostrophe) are accepted as an apostrophe and quotation mark, but there are some considerations:
If the intention of the text is to make it look as if it was traditionally typeset, then always use curly ones, as it was originally. Keep the straight one to denote prime.
Using the straight apostrophe and quotation can be a way of implying a digital mood to your document, the same way characters of the same width imply that the text was typed in a typewriter or console.
If the text is digital and to be consumed by human eyes, both (the curly and the straight apostrophe) are accepted as an apostrophe and quotation mark. Here schools differ and schism rules. We keep on with the expressive mess. Just be consistent within the same document.
If the intention of the text is to be semantically correct (as if you want a computer to be able to parse the semantics of the text), then you should pay close attention to Unicode's definition of the characters. U+0027 (the one that can be typed with the keyboard) is named "APOSTROPHE". Although that hints that it could be a good candidate to represent apostrophes, notice that Unicode is kind of ambiguos about this character indicating that, at least in English language, U+2019 is the preferred one for apostrophes. I personally think this goes against their principle of unification. You would expect that the apostrophe would be the same independently of the language in which it is used. Add to that the fact that they classify the apostrophe as "punctuation" which in French could be awkwardly accepted but in English makes no sense whatsoever. You are left, then, with the quandary of either using different apostrophes for different languages or using the controversial one for the English language. Discussions about which one is the correct one can get very passionate. On the other hand, it seems quite clear that U+2019 (and never the straight one) should be used as a right quotation mark. By the same token U+2032 should be used to denote prime, and U+02BC should be used to denote a modifier letter apostrophe (such as in the glottal stop of the Cockney pronunciation of bu'er or the transcription of "uh'oh").
Please notice that even if you select or type the correct character, the font you use might render it in a curly or straight fashion.