​

Input: mouse and keyboard

​

0. Source inventory (what feeds this area)

​

1. The event pipeline

Qt event
  → PyMOLGLWidget.mousePressEvent/mouseMoveEvent/wheelEvent   (pymol_gl_widget.py:178-196)
  → pymol2 SingletonPyMOL.button/drag                          (packages/engine/modules/pymol2/__init__.py:46,49)
  → _cmd._button / _cmd._drag                                  (packages/engine/layer4/Cmd.cpp:3626, 3646)
  → PyMOL_Button / PyMOL_Drag                                  (packages/engine/layer5/PyMOL.cpp:2908, 2890)
  → OrthoButton / OrthoDrag                                    (packages/engine/layer1/Ortho.cpp:2493, 2575)
  → Block::click/drag/release of the hit block                 (Ortho.cpp:2536 findBlock)
  → CScene::click → OrthoDefer(...) → SceneClick(...)          (packages/engine/layer1/Scene.cpp:4113-4155)

• All scene mouse handling is deferred. CScene::click/drag/release only push a lambda onto OrthoDefer (packages/engine/layer1/Scene.cpp:4113, 4118, 4129, 4146); the actual SceneClick/SceneDrag/ SceneRelease run later on the render/idle thread. when (a UtilGetSeconds timestamp) is captured at enqueue time, because all click/double-click timing uses it.
• PyMOL_Button multiplexes keys and mouse on the state argument (packages/engine/layer5/PyMOL.cpp:2896-2917): state == -1 → PyMOL_Key (ascii), state == -2 → PyMOL_Special (GLUT special code), state == 0 → mouse down, state == 1 → mouse up.
• Modifier mask is SHIFT=1, CTRL=2, ALT=4 (packages/engine/layer1/Ortho.h:20-22), built on the Qt side by get_modifiers (packages/engine/modules/pmg_qt/keymapping.py:44-58); note Qt Meta (⌘ on macOS) is folded into the same bit as Control (keymapping.py:51-52).
• Buttons are left=0, middle=1, right=2, wheel-forward=3, wheel-backward=4 (pymol_gl_widget.py:45-49, packages/engine/layer0/os_gl_glut.h:21-22), plus synthetic SINGLE_LEFT=100, SINGLE_MIDDLE=101, SINGLE_RIGHT=102, DOUBLE_LEFT=200, DOUBLE_MIDDLE=201, DOUBLE_RIGHT=202 (packages/engine/layer0/os_gl_glut.h:23-28).

​

2. Coordinate system: y-flip, framebuffer scale, reshape

def _event_x_y_mod(self, ev):
    pos = ev.position() if hasattr(ev, "position") else ev.pos()
    return (int(self.fb_scale * pos.x()),
            int(self.fb_scale * (self.height() - pos.y())),
            get_modifiers(ev))

1. Origin bottom-left. y_pymol = (widget_height_in_CSS_px - y_css) * dpr. Note the flip is applied to the logical height before scaling — reproducing this exactly matters for sub-pixel/off-by-one on non-integer DPR.
2. Device pixels. x and y are multiplied by fb_scale = devicePixelRatio (pymol_gl_widget.py:220).
3. fb_scale is also pushed into the engine as an integer setting: self.cmd.set('display_scale_factor', int(self.fb_scale)) (pymol_gl_widget.py:222). That drives _gScaleFactor (packages/engine/layer1/Setting.cpp:2946-2951) and hence DIP2PIXEL() (packages/engine/layer0/PyMOLGlobals.h:28-29), which sizes every internal-GUI hit rectangle (e.g. cControlBoxSize DIP2PIXEL(17) in packages/engine/layer1/Control.cpp:36, SceneScrollBarWidth DIP2PIXEL(13) in packages/engine/layer1/Scene.h:47). Changing it triggers OrthoCommandIn(G, "viewport") (packages/engine/layer1/Setting.cpp:2951).
4. Reshape is pymol.reshape(w, h, force) → _cmd._reshape → PyMOL_Reshape (packages/engine/layer5/PyMOL.cpp:2397-2405), which sets G->Option->winX/winY then OrthoReshape. resizeGL multiplies by fb_scale for QOpenGLWidget only (pymol_gl_widget.py:209-214); for the legacy QGLWidget path the scaling is done in paintGL via glViewport (pymol_gl_widget.py:202-206). The engine’s window size is in device pixels.

​

3. Ortho-level dispatch: blocks, grab, deferral

• Wheel events are suppressed while a real button is held: if button is scroll and I->ActiveButton is 0..2 and different, return immediately (:2503-2510).
• On P_GLUT_DOWN: I->ActiveButton = button; target block = I->GrabbedBy if set, else findBlock(x,y) (:2531-2540). findBlock iterates Blocks in reverse and calls recursiveFind (:2980-2990).
• On P_GLUT_UP: release is delivered to GrabbedBy and to ClickedIn (:2543-2556) — i.e. potentially twice. Faithful reimplementation should just forward the event and let the backend do this.
• Blocks attached: Scene (cOrthoScene, packages/engine/layer1/Scene.cpp:4243), and as cOrthoTool: ButMode (packages/engine/layer1/ButMode.cpp:575), Control (packages/engine/layer1/Control.cpp:862), Movie (packages/engine/layer1/Movie.cpp:56), Seq (packages/engine/layer1/Seq.cpp:710), Wizard (packages/engine/layer1/Wizard.cpp:835), Executive/object panel (packages/engine/layer3/Executive.cpp:16671), PopUp menus (packages/engine/layer4/PopUp.cpp:263), plus hidden Pop (packages/engine/layer1/Pop.cpp:71).
• OrthoGrab/OrthoUngrab (:1191, :1215) implement pointer capture. OrthoFakeDrag (:305-310) replays the last drag at LastX/LastY/LastModifiers — used for timing-driven pop-ups; it is triggered from Python via cmd._fake_drag (packages/engine/modules/pymol/cmd.py:171, packages/engine/layer4/Cmd.cpp:540).

​

4. The button→action table (ButMode)

​

4.1 Table shape

​

4.2 ButModeTranslate(G, button, mod) (packages/engine/layer1/ButMode.cpp:603-757)

• L/M/R → base 0/1/2, then +3 Shift, +6 Ctrl, +9 CtSh, +68 Alt, +71 AltShift, +74 CtrlAlt, +77 CtrlAltShift (:730-756).
• Wheel → slot 12..15 by modifier, then re-mapped by direction (:617-678): Slab → ScaleSlabExpand/Shrink; MovS → MoveSlabForward/Backward; MvSZ → MoveSlabAndZoomForward/Backward; IMSZ → inverted; MovZ → ZoomForward/Backward; IMvZ → inverted. Anything else returns -1 (wheel does nothing).
• Single/Double → base 16..21, then +6 Shift, +12 Ctrl, +18 CtSh, +24 Alt, +30 AltShift, +36 CtrlAlt, +42 CtrlAltShift (:678-726).
• ButModeCheckPossibleSingleClick (:583-601) returns true iff the corresponding single slot is bound (>= 0).

​

4.3 Action codes and their on-screen 5-char labels

​

5. cmd.button() bit packing

button_code = {left:0, middle:1, right:2, wheel:3,
               double_left:4, double_middle:5, double_right:6,
               single_left:7, single_middle:8, single_right:9}      # :30-41
but_mod_code = {none:0, shft:1, ctrl:2, ctsh:3, alt:4, alsh:5, ctal:6, ctas:7}  # :44-53

if button_num < 3:                       # L/M/R
    but_code = b + 3*m           if m < 4 else b + 68 + 3*(m-4)     # :851-855
elif button_num < 4:                     # wheel
    but_code = 12 + m            if m < 4 else 64 + m - 4           # :856-860
else:                                    # single/double
    but_code = (16 + button_num - 4) + m*6                          # :862
_cmd.button(_COb, but_code, act_code)                               # :864

​

6. Mouse rings and mode cycling

• cmd.config_mouse(ring) sets button_mode=0, replaces the global mouse_ring, then calls cmd.mouse() (controlling.py:168-202).
• cmd.mouse(action) (controlling.py:609-686): 'forward'/'backward' step button_mode modulo ring length; 'select_forward'/'select_backward' step mouse_selection_mode in 0..6 (:637-646); a bare mode name jumps directly; negative button_mode encodes a mode outside the ring as -1 - index_into(mode_name_list) (:657-660, :670). After applying, it sets button_mode_name and calls cmd.button() for every row, then unpick() for non-editing modes or deselect() for editing modes (:679-680), and finally refresh_wizard() (:685).
• cmd.edit_mode(active) (controlling.py:688-717) is the legacy toggle between *_viewing and *_editing for the current family. Used by the Builder panel (packages/engine/modules/pmg_qt/builder.py:1341).
• Display names (controlling.py:206-217): 3-Button Lights, 3-Button Maestro, 3-Button Viewing, 3-Button Editing, 3-Button Motions, 2-Button Viewing, 2-Btn. Selecting, 2-Button Editing, 2-Button Lights, 1-Button Viewing.
• mode_name_list order is load-bearing (controlling.py:219-232) — the comment says “okay to append new mode name, but don’t insert: order & position matter”, because button_mode persists negatively by index.
• The in-viewport ring cycler: clicking the ButMode block cycles modes; clicking its top two lines cycles the selection level instead; right-click opens the mouse_config menu; Shift or right-button or scroll-backward reverses direction (packages/engine/layer1/ButMode.cpp:147-188).
• mouse_config menu items (packages/engine/modules/pymol/menu.py:82-101): 3-Button Motions / 3-Button Editing / 3-Button Viewing / 3-Button Lights / 3-Button All Modes / (sep) / 2-Button Editing / 2-Button Viewing / 2-Button Lights.

​

7. Complete button × modifier → action matrices

​

7.1 three_button_viewing (:320-348) — the default UX

​

7.2 three_button_editing (:349-377)

​

7.3 three_button_motions (:378-407)

​

7.4 three_button_lights (:235-262)

​

7.5 three_button_maestro (:291-319)

​

7.6 two_button_viewing (:408-435)

​

7.7 two_button_selecting (:436-463)

​

7.8 two_button_editing (:464-491)

​

7.9 two_button_lights (:263-290) — reachable only via cmd.mouse('two_button_lights')

​

7.10 one_button_viewing (:492-534) — the only mode using alsh/ctal/ctas

​

7.11 default (:535-547) — mirrors PyMOL_SetDefaultMouse (packages/engine/layer5/PyMOL.cpp:2979-3021)

​

8. Click / double-click / single-click semantics

• Double-click (SceneClickCheckDoubleClick, :146-170): within cDoubleTime = 0.35 s (:26), within 10 px in both axes, same button → button is promoted to P_GLUT_DOUBLE_*. Only attempted if the corresponding single slot is bound or no modifier is held (:151).
• “Possible single click” (:664-674): set when ButModeCheckPossibleSingleClick or no modifier; also forced on if button_mode_name starts with '1' (i.e. 1-Button Viewing).
• On release (SceneRelease, :1141-1172): if the press→release gap exceeds 0.25 s + ApproxRenderTime, the single click is cancelled; otherwise state becomes 2 with a SingleClickDelay = 0.15 s. If the matching double slot is cButModeNone, the delay is zeroed so the single click fires immediately (:1165-1170).
• Deferred single-click dispatch (packages/engine/layer1/Scene.cpp:2439-2450): in SceneIdle, once now - LastReleaseTime > SingleClickDelay, SceneDeferClickWhen(..., LastButton + P_GLUT_SINGLE_LEFT, ...) synthesises the single-click event.
• Drag cancels single click: >0.15 s since press (SceneDrag, :1222-1227) or >4 px total movement (:2028-2035).
• Threshold for torsion picks: cButModePkTorBnd sets I->Threshold = 3 px so the first 3 px of movement do not drag (:909-913, consumed at :1461-1466 and :1514-1519).

​

9. Drag semantics per action (SceneDrag, packages/engine/layer1/SceneMouse.cpp:1205-2036)

• rota (RotXYZ) — virtual trackball, scale = 0.45 * min(W,H) (:1312-1315). virtual_trackball (default 1, packages/engine/layer1/SettingInfo.h:414) selects three formulations (:1774-1804): 2 = twist-aware relative-to-origin, 1 = classic centre-relative, 0 = pure delta. Rotation magnitude is mouse_scale * 2 * 180 * asin(|n1×n2|)/π, clamped by mouse_limit * |v1-v2| / scale (:1840-1850). mouse_scale default 1.3, mouse_limit default 100 (SettingInfo.h:296-297).
• rotz/irtz — in-plane twist omega about axis2 (:1861-1865, :1886-1904); irtz uses (LastX - x)/2 degrees about Z.
• move (TransXY) — v2 = (dx, dy) * SceneGetExactScreenVertexScale(origin) (:1725-1760), then roving_origin / roving_detail follow-ups.
• movz (TransZ) — factor = mouse_z_scale * dy/400 * max(5, -pos.z); sign flips unless legacy_mouse_zoom (:1905-1924). Defaults: mouse_z_scale 1.0 (SettingInfo.h:719), legacy_mouse_zoom 0 (:542).
• clip/clpn/clpf — near/far shifted by dx/10, dy/10 (:1925-1957).
• rotl/movl/mvzl — mutate the light/light2… setting vector by 0.01 * d (:1958-2022).
• Object/fragment/view drags (roto/movo/mvoz/rotv/movv/mvvz/rotf/movf/mvfz/torf/mova/mvaz/pktb) route to EditorDrag, ObjectMoleculeMoveAtom, ObjectMoleculeMoveAtomLabel, ObjectSliceDrag, ObjectDistMoveLabel (:1499-1724). Label picks (cPickableLabel) are re-targeted to the atom for object/fragment/view modes (:945-962).
• Gadget drag (color ramps etc.) is handled under cButModePickAtom (:1319-1367) and again in the object branch (:1522-1564); it uses RenderContext::UnitWindow vs Camera scaling.
• Movie/TTT capture: movie_auto_store grabs the object and sets ReinterpolateFlag (:828-843, :964-980, :1603-1614, :1189-1201).
• SceneNoteMouseInteraction (:37-43) aborts view animation and optionally restarts the frame timer (mouse_restart_movie_delay, default 0, SettingInfo.h:499).

​

10. Wheel / scroll

• Qt: wheelEvent → get_wheel_button → 3 (up) or 4 (down), then a synthetic down+up pair is sent (pymol_gl_widget.py:190-196).
• get_wheel_delta (keymapping.py:100-122) ignores horizontal scroll unless Shift is held, in which case horizontal delta is used (emulating shift-wheel = horizontal).
• Wheel actions are direction-resolved in ButModeTranslate (§4.2) and executed in SceneClick (SceneMouse.cpp:717-802): slab scale ±0.2*mouse_wheel_scale, slab move ±0.1*mouse_wheel_scale, zoom ±0.1*mouse_wheel_scale * (front+back)/2. mouse_wheel_scale default 0.5 (SettingInfo.h:623).
• Wheel over the ButMode block cycles mouse modes backwards (ButMode.cpp:159-160).

​

11. Pinch-zoom gesture

• On GestureStarted, snapshot pinch_start_z = cmd.get_view()[11].
• RotationAngleChanged → cmd.turn('z', last - current).
• ScaleFactorChanged → z = pinch_start_z / totalScaleFactor; view[11] = z; view[15] -= delta; view[16] -= delta (front/back clip follow the origin); cmd.set_view(view). There is an explicit workaround for totalScaleFactor == 1.0 (QTBUG-48138).

​

12. Box (rubber-band) selection

• Entered from cButModeRectAdd/RectSub/Rect/SeleAddBox/SeleSetBox/SeleSubBox → SceneLoopClick (SceneMouse.cpp:45-57), which seeds LoopRect, sets LoopFlag, calls OrthoSetLoopRect(G, true, rect) (packages/engine/layer1/Ortho.cpp:253-260) and OrthoGrab.
• Drag updates right/bottom only (:59-66) — so the rect is drawn from the anchor.
• Release normalises the rect, calls ExecutiveSelectRect(G, rect, mode) (:68-91).
• ExecutiveSelectRect (packages/engine/layer3/Executive.cpp:7432-7586) runs SceneMultipick (packages/engine/layer1/ScenePicking.cpp:332) over the rect, creates _tmp_rect_sele, then set/add/subtract against the active selection using the current sel_mode_kw, honouring log_box_selections.
• The rubber band itself is drawn by Ortho, so in the web port it becomes a React/CSS overlay — but the result must still come from the backend SceneMultipick.

​

13. Picking and selection semantics

​

13.1 Picking mechanism

​

13.2 Selection levels (mouse_selection_mode)

​

13.3 What each pick action does

• sele (SeleSet) / +/- (SeleToggle) — build sel_mode_kw(obj\index)andSelectorCreateinto the active selection name fromExecutiveGetActiveSeleName; toggle uses the symmetric-difference expression at :99-101. Honours auto_hide_selectionsandauto_show_selections (:131-134), then WizardDoSelect (:135`).
• pk1 (PickAtom1) — creates pk1, activates the editor with pkresi=1, logs cmd.edit("...",pkresi=1) (:404-429).
• pkat (PickAtom) — multi-atom editor picking pk1..pk4 via EditorGetNextMultiatom; clicking an already-picked atom unpicks it (:430-471).
• pkbd/pktb — SceneClickPickBond (:490-549) creates pk1+pk2 from the bond, logs cmd.edit("a","b"), and for pktb prepares a torsion drag.
• orig — ExecutiveOrigin at the atom (:282-310); cent — ExecutiveCenter (:311-330).
• drgm / drgo — issue cmd.drag("bymol (...)") / cmd.drag("byobject (...)") (:264-281); cmd.drag is defined at packages/engine/modules/pymol/editing.py:1020.
• menu — MenuActivate2Arg(..., "pick_sele", name, name) if the atom is in the active selection, else "pick_menu" (:382-403); with nothing picked, MenuActivate3fv(..., "main_menu", LastClickVertex) (:883-887). Menu contents live in packages/engine/modules/pymol/menu.py:1682 (main_menu), :1709 (pick_sele).
• clik (SimpleClick) — no selection side-effects; calls PyMOL_SetClickReady(name, index, button, mod, x, ScreenHeight-(y+1), pos, state+1, bond) (:1044-1058) or with an empty name if nothing was hit (:586-589). Note the second y-flip here — the click callback reports y in top-left origin.
• Nothing picked with sele clears the active selection to none; with +/- it disables it (SceneClickPickNothing, :558-597). Note the missing break at :573 — SeleSet falls through into SeleToggle, so a miss both clears and disables. Preserve or fix deliberately.

​

13.4 Click-ready callback (for clik)

​

13.5 Selection indicator (the pink dots)

​

14. Non-scene mouse targets inside the viewport

• Movie control bar (packages/engine/layer1/Control.cpp): 9 buttons, NButton = 9 (:62), hit-test which_button (:243-255). Release actions (:288-385): 0 rewind, 1 back, 2 stop (also clears sculpting and rock), 3 play/pause (Ctrl → rewind+play), 4 forward, 5 ending (Ctrl → middle), 6 toggle seq_view (label “S”), 7 toggle rock, 8 full_screen (label “F”). Buttons 3/6/7 render in an “active” colour when engaged (:645-649). The left nub drags the internal GUI width (:257-286), and a double-click within 0.35 s collapses/ restores it (:448-469).
• Scene buttons / scrollbar inside the scene block (SceneMouse.cpp:179-223, :642-709, :1233-1303): left = activate scene with interpolation, middle = rapid browse (Ctrl disables animation), right = drag-to-reorder or scene_menu pop-up; dragging reorders via cmd.scene_order([...]).
• Wizard block (packages/engine/layer1/Wizard.cpp:483+) — buttons and pop-ups.
• Object/Executive panel, Sequence viewer, Movie panel — other areas’ docs; they all arrive through the same OrthoButton path today.
• Spaceball / SDOF (packages/engine/layer1/Control.cpp:83-216, packages/engine/layer4/Cmd.cpp:3665 _sdof): 6-DOF queue, SDOF_NORMAL_MODE/SDOF_DRAG_MODE/SDOF_CLIP_MODE toggled by device buttons 1/2. Out of scope for a browser client unless WebHID is used — recommend explicitly dropping.
• Drag & drop of files onto the canvas (pymol_gl_widget.py:256-270) — accepts URLs, local files go to gui.load_dialog(url). Maps to HTML5 drag&drop + upload/path handoff.

​

15. Keyboard

​

15.1 Qt → PyMOL key codes (packages/engine/modules/pmg_qt/keymapping.py)

• special key → state = -2;
• otherwise state = -1, k = keyMap.get(key, -1), falling back to ord(ev.text());
• if still -1 and Ctrl held → k = key - 64 (Qt key codes are uppercase ASCII, so Ctrl-A→1);
• if Alt held → k = key (the raw uppercase key code);
• out-of-range (k > 255 or k < 0) is dropped.

​

15.2 C-side key routing

• mod == 4 (Alt) → cmd._alt(chr(k)), except '@' which is re-dispatched with no modifier (“option G produces ’@’ on some non-US keyboards”, :803-812).
• mod == 3 (Ctrl+Shift) → cmd._ctsh(chr(k+64)) (:854-855).
• Printable (k > 32 && k != 127) → inserted into the command line (add_normal_char, :821-838).
• 32 space — if the line is empty: presentation on → cmd.scene('','next'), Shift → rewind;mplay; otherwise mtoggle, Shift → rewind;mplay (:860-878).
• 127 delete, 8 backspace — line editing (:879-903).
• 1 Ctrl-A → beginning of line if arrows grabbed, else _ctrl('A') (:911-916).
• 5 Ctrl-E → end of line, else _ctrl('E') (:905-910).
• 4 Ctrl-D → delete char / filename completion query (:917-936).
• 9 Tab → PComplete tab-completion (Ctrl-I with Ctrl held → _ctrl('I')) (:937-958).
• 27 Escape → in presentation mode quits; else dismisses splash, else toggles text (Shift → toggles overlay) (:959-978).
• 13 Enter → parse current line; if empty and a movie panel exists: mview toggle / Shift mview toggle_interp / Ctrl mview toggle,freeze=1 / Ctrl+Shift mview toggle_interp,object=same; in presentation mode mtoggle (:979-1001).
• 11 Ctrl-K → truncate line (:1002-1013).
• 22 Ctrl-V → cmd.paste() if the line has text, else _ctrl('V') (:1014-1025).
• default → cmd._ctrl(chr(k+64)) (:1026-1028).

​

15.3 Python key dispatch (packages/engine/modules/pymol/internal.py)

​

15.4 Full default key binding table

​

15.5 Reserved / non-rebindable keys

​

15.6 Shortcut editor dialog (packages/engine/modules/pmg_qt/shortcut_menu_gui.py)

• Filter QLineEdit with placeholder “Filter”, live regex filtering (:79-84).
• Refresh QPushButton with refresh.svg icon, tooltip “Refresh the table to reflect any external changes” (:86-94).
• QTableView with 3 columns: Key, Command (click to edit), Description (:164-165). Key is read-only; editing a command calls cmd.set_key and marks the row “user defined” (:394-415). Deleted rows show "Deleted" (:174-177, :225-243).
• Buttons: Create New (:107-114), Delete Selected (:116-123), Reset Selected (:125-131), Reset All (:133-138), Save (:140-145).
• Sub-dialogs loaded from .ui forms: create_shortcut (fields keyEdit, commandEdit, createButton, helpButton), help_shortcut, change_confirm (confirmButton, cancelButton, doNotShowCheckBox) — :61-63, :277-286, :347-355.
• Key capture: eventFilter on keyEdit converts a live key event to PyMOL notation (keyevent_to_string :300-314, process_keyevent_string :316-342), mapping Control/Meta→CTRL, Control+Shift→CTSH, Alt→ALT, Shift→SHFT, and renaming PageUp→pgup, PageDown→pgdn, Home→home, Insert→insert, Up→up, Down→down, Left→left, Right→right, End→end (:32-41). Reserved keys are silently rejected (:288-290).
• Persistence: ~/.pymol/shortcuts_save.json (packages/engine/modules/pymol/save_shortcut.py:6, save_shortcuts :18-35, load_shortcuts_dict :37-54). Loaded at startup (pymol_qt_gui.py:418-419).
• Reconciliation logic to port as-is: ShortcutManager.check_saved_dict / check_key_mappings / reset_all_default / create_new_shortcut (shortcut_manager.py:23-139).

​

16. Precise browser-event mapping specification

​

16.1 Pointer events on the canvas

const BTN = {0: 0, 1: 1, 2: 2};             // pymol_gl_widget.py:45-49
const X = e => Math.round((e.clientX - rect.left) * dpr);
const Y = e => Math.round((rect.height - (e.clientY - rect.top)) * dpr);   // pymol_gl_widget.py:173-174
const MOD = e => (e.shiftKey?1:0) | ((e.ctrlKey||e.metaKey)?2:0) | (e.altKey?4:0); // keymapping.py:49-57

• Mouse tracking is always on in Qt (setMouseTracking(True), pymol_gl_widget.py:108) — passive moves are delivered even with no button down. OrthoDrag no-ops unless something is grabbed/clicked (Ortho.cpp:2588-2594), so sending every pointermove is correct but wasteful. So passive moves are sent only between pointerdown and pointerup, coalesced to the latest position per budget window and flushed before any button event (packages/viewport/src/input/coalescer.ts). The coalescer runs off a clock rather than requestAnimationFrame, because rAF stops dead in a hidden or occluded tab and an rAF-driven flush turns a whole drag into one jump at pointerup.
• e.button for pointerup is the released button; ensure a synthetic _button(b,1,…) is sent on pointercancel, blur, and visibilitychange so the backend never stays in a dragging state.
• e.getCoalescedEvents() should be ignored — PyMOL’s drag math is incremental (LastX/LastY), so replaying coalesced points would multiply rotation speed.
• Middle-click must preventDefault() on auxclick/pointerdown to stop autoscroll.
• Do not synthesise double/single clicks in the browser: PyMOL does that itself in SceneClickCheckDoubleClick / SceneIdle (§8). Browser dblclick must be suppressed.

​

16.2 Keyboard

specials = {ArrowLeft:100, ArrowUp:101, ArrowRight:102, ArrowDown:103,
            PageUp:104, PageDown:105, Home:106, End:107, Insert:108,
            F1:1 … F12:12}                            // keymapping.py:19-41
if (specials[e.key] !== undefined)  send _button(specials[e.key], -2, 0, 0, MOD)
else {
  k = {Escape:27, Tab:9, Backspace:8, Enter:13, Delete:127}[e.key]
      ?? (e.key.length === 1 ? e.key.codePointAt(0) : -1)      // keymapping.py:78-83
  if (k === -1 && (MOD & 2)) k = e.keyCode - 64                // keymapping.py:85-86  (Ctrl-A → 1)
  if (k !== -1 && (MOD & 4)) k = e.keyCode                     // keymapping.py:88-89  (Alt → raw code)
  if (k >= 0 && k <= 255) send _button(k, -1, 0, 0, MOD)
}

• e.keyCode is deprecated but is the only cheap source of the uppercase-ASCII code that keymapping.py relies on for Ctrl/Alt. Prefer deriving it from e.code (KeyA → 65, Digit0 → 48) to stay standards-compliant, and document the equivalence.
• Ctrl+letter and Alt+letter are browser/OS shortcuts (Ctrl-T new tab, Ctrl-W close, Alt-F menu…). preventDefault() recovers most but not Ctrl-W/Ctrl-T/Ctrl-N in most browsers. PyMOL binds CTRL-T (bond;unpick) and CTRL-F (wizard find), so those collide. The bindings stay as upstream defines them and are rebindable from the shortcut editor (apps/web/src/features/shortcuts/); the browser wins where preventDefault() cannot.
• macOS: Qt folds Meta (⌘) into the CTRL bit (keymapping.py:51-52), so e.metaKey → bit 2. The separate _cmmd path (internal.py:500-507, Ortho.cpp:775-786) is only reachable from the native macOS GLUT build and is not ported.
• Send on keydown only (Qt sends on keyPressEvent); ignore keyup. Ignore auto-repeat only if a binding is expensive — Qt does not ignore it.
• Tab must be preventDefault()-ed so it reaches the PyMOL command line for completion (mirrors pymol_qt_gui.py:449-455).

​

16.3 devicePixelRatio changes

1. resize the WebGL drawing buffer,
2. cmd.set('display_scale_factor', Math.round(dpr)) (pymol_gl_widget.py:222) — note it must be an integer or the C side warns and forces 1 (packages/engine/layer1/Setting.cpp:2953-2958),
3. _reshape(wDevice, hDevice, true). Wrap in try/catch: the Qt code notes set fails “with modal draw (mpng …, modal=1)” (pymol_gl_widget.py:223-225).

​

16.4 Pinch / trackpad zoom

• wheel with e.ctrlKey and no physical Ctrl pressed (track keydown/keyup state) → treat as pinch: on first event snapshot view[11] from cmd.get_view(), then z = startZ / totalScale; view[11] = z; view[15] -= (z - old); view[16] -= (z - old); cmd.set_view(view).
• Two-finger rotate → cmd.turn('z', deltaDegrees).
• If a real Ctrl key is down, fall through to _button(3|4, …) so mvsz still works.

​

17. Why the backend stays authoritative for the camera

• Picking is a GPU colour-pick pass in the backend’s own GL context (packages/engine/layer1/ScenePicking.cpp:17-38, PyMOLReadPixels at :149) — it renders with the backend’s camera. If the browser camera differs by one frame, clicks hit the wrong atom.
• Box select uses SceneMultipick over a screen rect (Executive.cpp:7432-7438) — same problem, amplified.
• Drag/edit math uses SceneGetExactScreenVertexScale and MatrixInvTransformC44fAs33f3f(I->m_view.rotMatrix(), ...) on the backend (SceneMouse.cpp:1490-1491, :1596-1597, :1729) — atom positions computed from mouse deltas depend on the backend’s rotation matrix and zoom.
• Clip planes, slab, roving detail, mouse_z_scale, virtual_trackball are backend settings applied inside SceneDrag; reimplementing them client-side means forking numerically sensitive code (SceneMouse.cpp:1762-2026).
• Scenes, mview, zoom animate=-1, rock, movie playback all animate the backend camera (ControlRock, Control.cpp:415-439; SceneIdle sweep, Scene.cpp:2410-2427), so the browser has to follow the backend regardless.

​

The two input paths that came out of this

​

ButModeGet/ButModeTranslate are mirrored, not exposed

​

18. The bridge surface for this area

• the mouse config table — mirrored in packages/viewport/src/input/butmode.ts (§17), not fetched;
• the loop rect for box select — OrthoSetLoopRect (Ortho.cpp:253) is only drawn internally, so the browser draws its own from the same press/current coordinates (apps/web/src/features/console/OrthoLoopRect.tsx);
• get_click_string — _cmd.get_click_string (Cmd.cpp:1420) has no cmd.* wrapper upstream, so the clik / SimpleClick pathway is reached through the raw _cmd entry point.

​

19. Constraints this area lives under

1. Picking requires the backend’s GL context. SceneDoXYPick renders a pick pass and calls PyMOLReadPixels (ScenePicking.cpp:149), so every pick costs a full render. A GL-less bridge cannot pick at all — packages/viewport/src/picking/route.ts chooses between the backend pass and a client-side ray on that basis.
2. Camera divergence breaks picking silently — the wrong atom is selected with no error. This is the reason §17 keeps the view authoritative on the backend.
3. Browser keyboard hijacking. CTRL-T, CTRL-F and CTRL-W-adjacent defaults collide with the browser and some cannot be preventDefault()-ed.
4. Middle-click and right-click need preventDefault on auxclick/contextmenu; some Linux browsers still paste on middle-click.
5. Ctrl+wheel is trackpad pinch in Chrome/Firefox, colliding with PyMOL’s Ctrl+wheel = mvsz (§16.4).
6. The button table can drift. The client mirrors mode_dict (§17); a plugin calling cmd.button directly changes ButMode’s real state without changing the mirror. The mirror tests pin it against the source, but they cannot see runtime writes.
7. DPR/y-flip off-by-one. Qt flips using the logical height before scaling (pymol_gl_widget.py:174); a naive (h_device - y_device) differs on fractional DPR and mis-picks edge pixels.
8. Latent upstream bugs a faithful port inherits: duplicate double_left row in three_button_motions (controlling.py:398-399), duplicate row in two_button_selecting (:456-457), missing break in SceneClickPickNothing (SceneMouse.cpp:573), double release dispatch in OrthoButton (Ortho.cpp:2543-2556), ButModeSet(cButModeMiddleCtSh) written twice in PyMOL_SetDefaultMouse (PyMOL.cpp:3005 — the source itself comments “SET TWICE?!?”).
9. SDOF / spaceball (Control.cpp:83-216, _sdof) has no browser equivalent short of WebHID and is not ported.
10. Deferred execution ordering. All scene input is queued through OrthoDefer; a transport that reorders or drops messages corrupts drag state, so input rides one strictly ordered connection with no parallel channels.
11. Timing-based semantics (0.35 s double-click, 0.25 s single-click window, 0.15 s delay) are measured on the backend against UtilGetSeconds, so transport jitter inflates the measured press-to-release gap. The client stamps when from the event itself, not from send time (packages/viewport/src/input/coords.ts), and the bridge passes that through.