How to use it
- Drop the screenshot or paste it with Ctrl+V. Drag on it to select just the text region – a tight crop is the single biggest accuracy win.
- Pick a preset: Dialogue box, Menu / inventory, HUD / scoreboard or Retro pixel font. Each sets the upscale, inversion, contrast, threshold and text layout.
- Check the processed preview: you want dark, crisp text on a plain light background. Adjust invert, contrast, threshold or sharpen until it looks like a printed page.
- Press Recognise text. The first run downloads the engine and the language model; later runs start immediately.
- Proofread the words marked in tomato, edit in the text box, then copy or download .txt / .json.
Example
A 1920 × 1080 screenshot of an RPG dialogue box with white 22 px text on a dark blue panel: select the box (about 1,400 × 220 px), choose “Dialogue box” – upscale ×2, grayscale, invert, contrast +30, automatic threshold (Otsu picks about 118 on this panel), layout “One block”. The engine receives a 2,800 × 440 px black-on-white image and returns three lines, 41 words at 93% page confidence; two words under 60% (a stylised name and an ellipsis) are marked for checking. The JSON lists each word with its box, e.g. { "text": "Welcome", "confidence": 96, "box": { "x": 412, "y": 731, "width": 118, "height": 24 } }, in the coordinates of the original screenshot.
How the preprocessing and recognition work
pipeline : crop → upscale (smooth or nearest ×1–×4; pixel font: nearest ≥ ×3 + 3×3 blur) → grayscale (Rec. 601 luma)
→ brightness / contrast (factor 259·(c+255) / (255·(259−c))) → unsharp mask (src + k·(src − blur), k ≤ 1.5)
→ threshold (Otsu or manual, 0–255) → invert
boxes : original x = region.x + x_processed ÷ scale (same for y, width, height)Recognition uses tesseract.js 6 (Tesseract 5, LSTM engine mode 1) in a web worker with the page-segmentation mode you pick: automatic (3) for mixed layouts, single block (6) for a cropped dialogue, single line (7) for a score or item name, sparse text (11) for HUD labels scattered across the screen. The structured output (blocks) gives every block, paragraph, line and word a bounding box and a 0–100 confidence; the tool rebuilds the text from the lines so breaks are preserved, scales the boxes back to the original screenshot and flags words under 60%.
Limits: Tesseract's models are trained on printed fonts. Clean UI fonts, subtitles, menus, scoreboards and dialogue read well after inversion; heavily stylised display fonts, outlined or glowing text, text over busy artwork, perspective text and very small fonts (under about 12 px before upscaling) are often wrong or missing – the confidence marks tell you where. One language model per run; mixed scripts need separate passes. The CDN download fails cleanly offline instead of guessing.
Sources: tesseract.js on GitHub (API, output formats, default CDN paths), Tesseract documentation – page segmentation modes and image preprocessing, Otsu, N. (1979) – A threshold selection method from gray-level histograms.
Questions
- Why does the preview look inverted or black and white?
- Tesseract expects dark text on a light background, like a printed page. Most game UI is the opposite, so the presets invert the image and often threshold it to pure black and white. What you see in the preview is exactly what the engine reads – if the text is not legible there, it will not be recognised.
- Which game fonts work?
- Regular sans-serif UI fonts, subtitles, chat, quest logs, inventories and scoreboards work well. Pixel fonts work with the Retro preset once each pixel is at least 3 screen pixels. Decorative title fonts, heavy outlines, glow effects, gradients and text over detailed backgrounds are unreliable; crop them out or expect to correct the result.
- What is in the JSON file?
- The edited text, the raw recognised text, the language, layout mode, region and scale, and the blocks → lines → words tree with a confidence (0–100) and a box {x, y, width, height} for each element in original screenshot pixels. It is ready for scripting, for example to align translations or to build a subtitle overlay.
- Is anything uploaded?
- No. The screenshot is cropped and processed on a canvas in your browser and handed to a web worker on the same page. The only network traffic is the one-time download of the engine and the language model from the jsdelivr CDN.