I was wondering, given the sheer amount of images with text in front, posted on Facebook, some of which may make you want to select and translate via machine translation of copy and paste, has anyone designed a compatible file format for images where you can do this (select, copy, paste, translate) the text.

Where can I find a command line program or API to extract the text from the picture and present it? How do I do this on Facebook? What are the steps?


  • 1
    Do you mean such as PSD, or XCF for example? These can store editable/selectable text layers. But they don't work in web browsers. It's possible to overlay an image with text using HTML and CSS in a web page, but that's not an image format - it's how the web page is encoded.
    – Billy Kerr
    Jan 12, 2019 at 15:32
  • There are many browser plug-ins that do OCR... for example Project Naptha for Chrome. Of course OCR is not the same as simply copying text as it is. If a letter is hard to recognize because it's not fully visible or something, it may come out wrong.
    – Joonas
    Jan 14, 2019 at 0:27

1 Answer 1


JPGs and other common bitmap images are enough. Google has implemented automatic image text translation application for common phones. Get it and let your phone watch & translate the image texts on your screen. It's well possible that you get sometimes something which makes sense, too.

I do not know how to get the same functionality to a normal PC, but Google Translate allows you to upload photos for translating the texts. Try it. Read Google's documentation, too.

ADD due the edits in the question and a comment. Actually Adobe's PSD would be ideal format, because text blocks can be ready to use editable texts. GIMP's XCF or SVG would be as useful. Even Windows metafiles (=WMF) can contain photos, vector drawing objects and editable texts.

One can meet some difficulties if he wants to persuade people to upload their images not as JPGs, but as PSDs, XCFs, SVGs or WMFs and definitely all image texts as text objects, not as raster images like texts in JPGs.

Some discussions with Mr Zuckerberg would be useful because he has more tools to persuade people to behave in a wanted way. For his paying customers he can also insert a special mode, which provides the image texts as ready to use extracted text files.

I am afraid that as long as Mr Zuckerberg hasn't got interested, you must try to extract texts from JPGs. A command line program for it must be quite capable because you cannot show to it easily does some text exist and which texts belong together. Your program must be able to detect text regions and recognize their contents without assistance.

The needed technology surely exists. As so many other really useful non-trivial applications, your program must be based on pattern matching. It's used to detect and recognize nearly anything which is partially covered and surrounded by clutter, but has something which is detectable and possible to classify. For ex. security service providers and police use it to spot interesting persons, cars, equipment and behavioural patterns.

Pattern matching is math. I bet 4 years university level studies are needed to understand properly existing works. That is few miles above my capablities, so I cannot give exact answers about details.

Do some searches. try:

  • detecting text in images
  • OCR

I met with them at first this: https://www.mathworks.com/help/vision/examples/automatically-detect-and-recognize-text-in-natural-images.html

  • I don't think it's enough. The text could be unreadable. Is there a better, more computer and image processing friendly solution? Jan 13, 2019 at 17:42
  • What if I want to write a command line program to extract the text from a picture, how do I do it? Jan 13, 2019 at 17:43
  • @JoselinJocklingson text in an image is no different than the pixels around it - a pixel is a pixel is a pixel. Nothing sees "type" other than humans. The only possible method would be Optical Character Recognition (OCR) -- which is MUCH more than a simple command line prompt.
    – Scott
    Jan 13, 2019 at 19:08
  • @JoselinJocklingson use tesseract
    – joojaa
    Jan 13, 2019 at 19:37
  • But why overload computers with OCR. Why not make these connotations processor-friendly by embedding the text as text in the image's file format, so that you can grep the image text as a string when opening the image via a text editor, perhaps exif tags.? Jan 14, 2019 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.