IssueRaiser

I created a basic similar app for my own use because I wanted to have an idea what people are conversing in russian in online lobbies.

I created a basic similar app for my own use because I wanted to have an idea what people are conversing in russian in online lobbies. WPF, Tesseract OCR and Microsoft's translation API. https://streamable.com/ykng5u A fun side project that I do end up using a bit. Gonna bind the capture to some hotkey so I can use it without changing app focus. Most annoying problem though is that Tesseract OCR often gets confused when you make it read combined latin+cyrillic letters and the font isn't something Tesseract prefers. Especially when there's something behind the text. Kind of disappointed that the most popular API often has a lot worse results than a human would just transcribing the letters. Wouldn't be surprised if OCR software would leap soon due to a product similar to Whisper. Comes to mind that the best possible app that does this would be kind of like the old "word lens" iPhone application but on all screens, meaning it would replace text from the raw screen input with text of another language, while keeping the appearance/color/scale/rotation of the original text. This would free it from needing to be built-in to whatever UI library is producing the text, and would work on recorded video too. Immediate latency/performance problems come to mind though but could be a fun thing to try.

Posted by maxlin on Apr 28, 2023, 3:39 AM

Tags

Related issues

Raise real-world problems, find people facing the same issue, and discuss solutions.2,500 issues0 comments

ux (SaaS)

I created a basic similar app for my own use because I wanted to have an idea what people are conversing in russian in online lobbies.

maxlinApr 28, 2023, 3:39 AM

I created a basic similar app for my own use because I wanted to have an idea what people are conversing in russian in online lobbies. WPF, Tesseract OCR and Microsoft's translation API. https://streamable.com/ykng5u A fun side project that I do end up using a bit. Gonna bind the capture to some hotkey so I can use it without changing app focus. Most annoying problem though is that Tesseract OCR often gets confused when you make it read combined latin+cyrillic letters and the font isn't something Tesseract prefers. Especially when there's something behind the text. Kind of disappointed that the most popular API often has a lot worse results than a human would just transcribing the letters. Wouldn't be surprised if OCR software would leap soon due to a product similar to Whisper. Comes to mind that the best possible app that does this would be kind of like the old "word lens" iPhone application but on all screens, meaning it would replace text from the raw screen input with text of another language, while keeping the appearance/color/scale/rotation of the original text. This would free it from needing to be built-in to whatever UI library is producing the text, and would work on recorded video too. Immediate latency/performance problems come to mind though but could be a fun thing to try.

Comments (0)

No comments yet. Start the discussion.

Comment on this issue