I was wondering, given the sheer amount of images with text in front, posted on Facebook, some of which may make you want to select and translate via machine translation of copy and paste, has anyone designed a compatible file format for images where you can do this (select, copy, paste, translate) the text.
Where can I find a command line program or API to extract the text from the picture and present it? How do I do this on Facebook? What are the steps?
Thanks.
JPGs and other common bitmap images are enough. Google has implemented automatic image text translation application for common phones. Get it and let your phone watch & translate the image texts on your screen. It's well possible that you get sometimes something which makes sense, too.
I do not know how to get the same functionality to a normal PC, but Google Translate allows you to upload photos for translating the texts. Try it. Read Google's documentation, too.
ADD due the edits in the question and a comment. Actually Adobe's PSD would be ideal format, because text blocks can be ready to use editable texts. GIMP's XCF or SVG would be as useful. Even Windows metafiles (=WMF) can contain photos, vector drawing objects and editable texts.
One can meet some difficulties if he wants to persuade people to upload their images not as JPGs, but as PSDs, XCFs, SVGs or WMFs and definitely all image texts as text objects, not as raster images like texts in JPGs.
Some discussions with Mr Zuckerberg would be useful because he has more tools to persuade people to behave in a wanted way. For his paying customers he can also insert a special mode, which provides the image texts as ready to use extracted text files.
I am afraid that as long as Mr Zuckerberg hasn't got interested, you must try to extract texts from JPGs. A command line program for it must be quite capable because you cannot show to it easily does some text exist and which texts belong together. Your program must be able to detect text regions and recognize their contents without assistance.
The needed technology surely exists. As so many other really useful non-trivial applications, your program must be based on pattern matching. It's used to detect and recognize nearly anything which is partially covered and surrounded by clutter, but has something which is detectable and possible to classify. For ex. security service providers and police use it to spot interesting persons, cars, equipment and behavioural patterns.
Pattern matching is math. I bet 4 years university level studies are needed to understand properly existing works. That is few miles above my capablities, so I cannot give exact answers about details.
Do some searches. try:
I met with them at first this: https://www.mathworks.com/help/vision/examples/automatically-detect-and-recognize-text-in-natural-images.html
