EditHere: turning UI feedback into JSON an AI coding tool can apply

The hard part of prompting an AI is saying where

EditHere starts from a specific frustration, and the README states it in the language of a bug report rather than a product pitch. When you work with an AI on an interface, the intent is usually obvious, such as moving the button in the top right a bit further left, making the card bigger, or noting that the colour is wrong, and the expensive part is describing the position and the scope in words, while the AI still frequently modifies the wrong place.

EditHere's answer is to put the instruction back into the picture. You circle an area, you write a note, you drag a component, or you pause a video and write on the frame, and the application then packages the original image, the annotation coordinates and the layout changes into a single JSON document that can be handed to Codex, Claude Code, WorkBuddy, Qcode or any other AI tool.

The comparison table in the README is the clearest summary of the design. Locating something becomes a click precise to the pixel instead of a phrase like the button in the top right, slightly lower. A layout adjustment becomes a drag and scale that records the real before and after coordinates instead of an approximate nudge. Multiple edits become numbered notes, each tied to a position, instead of one long paragraph that is easy to miss half of. A video or animation problem becomes a note on the exact frame, with the timestamp and a frame capture attached. And handing it over becomes the AI opening the canvas through a command line tool and waiting for you to press a single button.

The stated scope is broad: web and application interfaces, game HUDs, data charts and design review, in short anything where the conversation needs to be about where and how rather than about what.

Four ways to point at something in a screenshot

The annotation model has four modes, and the choice between them is the difference between a note that points somewhere and one that points at exactly one thing.

Smart selection is the first. Click an element that has been detected and you can write a note against it, which works because the application identifies interface elements as you hover. Point annotation targets a precise pixel, so a note can say something about a specific feature of a specific pixel rather than about a region. Box annotation drags out an arbitrary area, which is the one you want when the complaint is about a layout region rather than a control. A global comment applies to the whole frame, for the remarks that are about the composition rather than about an element.

Each note carries a number and is paired with a position in the image, and the export at the end is one JSON document. The structure is small and stable. The top level holds an optional base64 PNG of the original image, a field naming the annotation space, and an array of objects. Each object has a source rectangle with x1, y1, x2 and y2 coordinates, a movements array, and an annotations array of the note strings. A global comment is the same shape with a null source, and a layout move is the same shape with an entry in movements holding the destination rectangle.

  "annotationSpace": "result",
  "objects": [

The complete field list is in the user guide under the feedback JSON section, and the command reference with examples is in the agent CLI document. The coordinates are what make the document useful, since a tool reading it can locate the region rather than guess at it.

Explode turns a screenshot into a layout diff

The feature called big bang is the one that goes furthest from a plain annotation tool, and it is the answer to the instruction that means move this thing over there.

One click breaks the captured screen into movable components. You drag them, scale them proportionally, or type exact coordinates, and every movement is automatically recorded as an annotation holding the real region before and after the move. The original position is left empty, with the README describing it as space for the AI to fill in with whatever belongs there.

The reasoning is worth spelling out, because it explains why this is more than a drawing tool. When you describe a layout change in words, the AI has to interpret an intention, and the description is where the information loss happens. A drag produces a before and after rectangle for the same element, which is a measurement rather than an intention, and the empty original position tells the receiving tool that something was removed from there rather than merely left alone.

That is the same idea as the rest of the application, taken one step further. Instead of a note about position, the artefact is a position, and the model has nothing to interpret. The cost is that explode depends on the screen having been successfully decomposed into components in the first place, so it is a feature for interfaces with recognisable elements and less useful for a canvas or a game scene where the decomposition has no meaning.

Screenshots that carry measurements, not just pixels

The capture layer does considerably more than grab the screen, and the details are what separate a usable reference from a picture.

A global hotkey freezes the screen, and elements are recognised as you hover. The scroll wheel changes the size of the detected region, and you can also drag a box by hand. Once a selection exists, a toolbar appears above it with options for annotation, text recognition, long screenshot, pasting an image, saving and copying, and a style column on the right for corner radius, shadow and border, rendered as you adjust, with the window able to stay on top and several such windows pinned at once.

Around that there is a set of precision details. A magnifier follows the pointer during a drag and shows pixel coordinates along with RGB, HEX, HSV and HSL readings, and pressing the C key enters colour picking and copies the value. The arrow keys move the pointer one pixel at a time, and Shift or Ctrl with an arrow shrinks or grows the selection by a single pixel. Ratios can be fixed at 1:1, 4:3, 16:9, 9:16 or a custom value, and the size can be typed in exactly. The last 20 screenshots and 10 selections persist across sessions, you page through the history with the angle bracket keys, and R restores the previous selection. Pasted images can be scaled, rotated, flipped, given transparency and a shadow, and the first item in the context menu sends the image straight into the editor as an annotation.

The practical value of the magnifier and the pixel stepping is that a design note about a one pixel offset becomes expressible, and a note about a colour becomes a copyable value rather than a description of a colour.

Text recognition and long screenshots without touching the network

Two features are worth separating because they solve problems that usually send you to a hosted service.

Text recognition works on a region you select and lists the recognised text line by line. Clicking a line highlights the corresponding position on the original image, and you can copy all of it or a single line. The implementation detail is the important one: it uses the operating system's own engine, the Windows OCR on Windows and Vision on macOS, and the README states plainly that it does not go online, does not upload and adds no dependency. For a screenshot of an internal document, that is the difference between a feature you can use and a feature your security team will ask about.

Long screenshot does the opposite of a normal capture. You select a scrolling area, click long screenshot, and the application scrolls and stitches while a thumbnail preview grows with the content, keeping a fixed header bar only once instead of repeating it on every segment. It works in both directions, scrolls automatically, crops with one click, and hands the result to the annotation stage.

Both features are local by design, and the README makes the wider claim that screenshots, element recognition, text recognition and editing all happen on your machine and that EditHere does not upload your screen. That is the local-first promise, and it holds for the parts the application implements itself. It is worth separating from the AI step, which by definition sends the exported document somewhere.

Video annotation attaches a timestamp and a frame to every note

The video mode extends the same idea to time. You open a video, play it or drag the timeline, pause on the frame that needs changing, and annotate it. You can then continue playing and annotate at another moment, and the timeline keeps labelled markers so you can jump back to any frame you have already commented on.

The export follows the same rule as the screenshot case. Each note in the JSON carries a timestamp and a frame capture, which is what makes the document usable for animation, games and interaction walkthroughs, where a description like around the third second is useless and a timestamp plus an image is not.

The format is documented separately in the video annotation document in the docs directory, and the repository also has a project save mechanism, where an .edithere project embeds the original image, the annotations and the edit state so a session can be reopened later. Video projects reference the source file and store the annotated frames rather than the video itself, which keeps a project folder manageable.

Input handling is broad. You can screenshot, open, drag in or paste PNG, JPEG, WebP and BMP images, and MP4, MOV and WebM video. The interface has light and dark themes, Simplified Chinese and English, and 22 shortcuts plus the bottom toolbar are all customisable.

The handoff to an AI, and the licence you inherit

The last piece is the handoff, and the design decision here is that EditHere calls no model. It does not know anything about which AI you use, and the README is explicit that the application's job is only to organise the feedback.

There are two routes. Manually, you copy the JSON content, the JSON file or an annotated image and paste it into any AI. Automatically, the repository ships an edithere skill and an edithere-cli command, so the AI runs edithere-cli annotate, which opens the canvas and waits; you annotate, press the button that returns to the AI, and the tool receives the feedback and continues. Cancelling or timing out does not turn your unsubmitted edits into change requests, which is a small detail that prevents a whole class of confusing behaviour.

Installation has a similar shape, and it is unusual enough to be worth knowing. Rather than a command, the README gives a prompt you copy into an AI tool that has terminal and file access, such as Codex or Claude Code, and that prompt tells the tool to follow the AI setup document, to prefer reusing an already installed or fully extracted program, to install only if needed, to verify the CLI works and the skill is where the tool can see it, and to tell you how to start your first annotation, including any system permission you have to grant. A second prompt does the same for a portable Windows build, explicitly asking the tool not to run an installer and not to modify startup items or the PATH. The README notes that the AI tool needs local terminal and file permissions, and that a web chat window alone cannot install a local program.

The licensing is where a company has to stop and read. The badge names PolyForm Noncommercial 1.0.0, and the repository also carries COMMERCIAL-LICENSE.md, LICENSING.md, NOTICE, TRADEMARK-POLICY.md and a CLA.md, which together describe a split between noncommercial use, a commercial grant and the trademark rules. The project is written in C++ with CMake, and the default branch is codex/native. The releases are frequent, with v0.11.1 and v0.11.0 both on 2026-10-13 and v0.10.5 on 2026-10-10, and the last push was on 2026-10-20 with the repository not archived.

Editorial conclusion

EditHere fits a developer or designer working with an AI coding tool on a user interface, where the bottleneck is describing where to change rather than what to change, and it does not fit a commercial deployment without reading LICENSING.md and COMMERCIAL-LICENSE.md, since the badge names PolyForm Noncommercial 1.0.0. Before adopting it, run one annotation on a real screen and inspect the exported JSON, because the field names in that file are the contract your AI tool has to understand.

Frequently asked questions

Official sources

1. Inginnng/EditHere on GitHub
2. Issues
3. README
4. Releases