Spike 02 — Headless feedback capture
Status: BLOCKER RESOLVED. Full PyMOL console parity is achievable headless, with zero C++ changes, provided the bridge does two specific things. Both are non-obvious and both are the opposite of what the current build/architecture docs assume.STATUS — re-verified on 2026-08-02: STILL TRUE, IN FULL
This is the spike that has aged best. §1’s two rules are implemented verbatim inpackages/bridge/tenmol_bridge/engine.py, with this file’s reasoning in the comments beside them:options.no_gui = 0at:125(with-cexplicitly rejected at:121-123),options.internal_gui = 0/internal_feedback = 0at:129-130,pymol2.SingletonPyMOL()at:142under a comment that repeats §1 rule 2, and_install_pcatch()at:162. §9’s ordering trap — uninstallpcatchbeforep.stop(), becauseSingletonPyMOLGlobalsis nulled first — is at:404-407and cites “spike 02 §9”. §9’sFeedbackCapturewas a reference implementation; the shipped code istenmol_bridge/feedback.py+engine.py, not a copy of it. The behaviours §4 and §8 pin — destructive single-consumer read,None(not[]) on a lock miss, the unbounded queue, the ~1018-char hard split, the line-prefix classification table — are the reason that module has the shape it has. Two citation fixes, made in place below: §11 items 1 and 2 pointed at an"action item (3)"of00-build.mdthat has never existed; the finding they mean is00-build.md§5.1/§5.3 and its recommendations are §7.
1. The answer in two rules
Rule 1 — never pass-c. OrthoFeedbackIn() is gated on G->Option->pmgui
(packages/engine/layer1/Ortho.cpp:492-499). pmgui is set from !options.no_gui (packages/engine/layer1/P.cpp:1820), and
-c sets no_gui=1 (packages/engine/modules/pymol/invocation.py:401). Start with no_gui = 0 and suppress the
GUI by simply never creating a GL context and never calling _draw.
Rule 2 — use pymol2.SingletonPyMOL, not pymol2.PyMOL. The pcatch stdout hook writes
through the file-scope SingletonPyMOLGlobals pointer (packages/engine/layer1/P.cpp:2667). With a non-singleton
instance that pointer is nullptr, so pcatch silently discards every print() — worse than
not installing it. pmg_qt already does the right thing: packages/engine/modules/pmg_qt/pymol_gl_widget.py:3
is from pymol2 import SingletonPyMOL as PyMOL.
HaveGUI is only assigned from pmgui inside PyMOL_DrawWithoutLock()
(packages/engine/layer5/PyMOL.cpp:2244-2248), which a headless bridge never calls. So no_gui=0 costs nothing:
ray, png, refresh, idle, scenes, movies and reshape all still work with no GL context
(verified, §6).
2. Route-by-route results
Every row below was executed. “user-visible” = the exact text the Qt feedback pane shows.2a. The -cq blocker, reproduced (e3_cq.py)
_get_feedback() is empty every single time. pcatch was installed
here and did not help — the gate is upstream of it, in OrthoFeedbackIn.
2b. The working route (e2_pcatch.py)
iterate line: the user’s print output (N ALA) is correctly interleaved between
the prompt echo and the C-level summary. That interleaving is the reason to use pcatch rather
than a separate Python-side tee — both streams pass through the same OrthoAddOutput line buffer.
2c. The non-singleton trap (e4_nonsingleton.py)
PCatchWrite’s if(SingletonPyMOLGlobals) guard.
3. How the Qt GUI does it today
packages/engine/modules/pmg_qt/pymol_gl_widget.py:3—from pymol2 import SingletonPyMOL as PyMOLpackages/engine/modules/pmg_qt/pymol_gl_widget.py:99-105—self.pymol.start()thenimport pcatch; pcatch._install()packages/engine/modules/pmg_qt/pymol_qt_gui.py:391-394— aQTimer(single-shot, 100 ms initially)packages/engine/modules/pmg_qt/pymol_qt_gui.py:941-958—update_feedback():feedback = self.cmd._get_feedback()→colorprinting.text2html('\n'.join(feedback))→browser.appendHtml(...)→feedback_timer.start(500)packages/engine/modules/pmg_qt/pymol_qt_gui.py:964— after each typed command,feedback_timer.start(0)for an immediate drain.
_draw. The
web bridge is a 1:1 translation: replace QTimer with a poller thread / asyncio task, replace
text2html + appendHtml with an SSE or WebSocket stream of JSON line records.
colorprinting.error/warning/suggest are plain print in open source
(packages/engine/modules/pymol/colorprinting.py:29-32), and colored_feedback reports
" Setting-Warning: colored_feedback is not supported in Open-Source version of PyMOL" — so
feedback lines contain no ANSI escapes and the React client needs no ANSI parser. (Belt and
braces: OrthoFeedbackOut strips ANSI when colored_feedback is off, packages/engine/layer1/Ortho.cpp:502-516.)
4. Consume-once semantics — CONFIRMED DESTRUCTIVE
packages/engine/modules/pymol/internal.py:593-606 loops _cmd.get_feedback() until it returns empty;
packages/engine/layer4/Cmd.cpp:3892 calls OrthoFeedbackOut, which does front() + pop()
(packages/engine/layer1/Ortho.cpp:502-516). Measured (e5b.py):
- Exactly one consumer per process. If the HTTP handler and a background poller both call
_get_feedback(), lines are split randomly between them. Route everything through one owner. - PyMOL keeps no scrollback for you. (
I->Line[]is a 256-entry ring,OrthoSaveLines 0xFFatpackages/engine/layer1/Ortho.cpp:62, but it is not readable from Python.) The bridge must own the ring buffer so a reloading browser tab can replay history. - The queue is unbounded until drained. 5000
cmd.do('print(...)')calls with no drain produced a single_get_feedback()returning 10000 lines with nothing dropped (e5_semantics.py). Good for correctness, but it is a memory leak if the poller ever stops. Keep polling. - It can return
None, not[].internal._get_feedbackreturnsNonewhenlock_attemptfails. Observed 1Nonein ~100 polls while the main thread was rendering (e6_thread.py).if not fb:treatsNoneand[]identically and is fine for skipping, but never dolines.extend(fb)without theNonecheck.
5. cmd.feedback() interaction — a genuine trade-off
The terminal printf in OrthoNewLine (packages/engine/layer1/Ortho.cpp:1160-1169) is gated on
Feedback(G, FB_Python, FB_Output). So is PCatchWrite (packages/engine/layer1/P.cpp:2668). Measured
(e14_mute.py), after cmd.feedback("disable","python","output"):
- Muting stops the duplicate echo on the launching terminal and keeps C-origin lines in the queue…
- …but it kills Python-origin capture entirely.
python/output enabled and accept that the terminal PyMOL was launched
from also shows the console text. For a local desktop-replacement that is a feature, not a bug.
Per-module verbosity control (cmd.feedback("disable","executive","actions") etc.) works normally
and affects the queue as expected — verified in e5_semantics.py.
6. no_gui=0 does not break headless operation
e9_pipeline.py / e10_opts.py, SingletonPyMOL with no_gui=0, no GL context anywhere:
no_gui=0, internal_gui and internal_feedback default to 1 and PyMOL reserves
screen real estate for its own overlay — reshape(800,600) yielded a viewport of (580, 582),
not (800, 600). Setting internal_gui = 0 and internal_feedback = 0 before start() restores
the exact viewport without affecting feedback capture (e10_opts.py):
OrthoFeedbackIn only reads pmgui; internal_feedback is used solely to decide whether to mark
the ortho layer dirty (packages/engine/layer1/Ortho.cpp:1119-1122).
7. Fallback route (documented, not recommended): fd-level dup2
e8_fd.py — pipe over fd 1 + reader thread, under -cq:
-c. Reject it anyway: it needs a dedicated
reader thread or the 64 KiB pipe buffer will deadlock PyMOL mid-render; it swallows the process’s
real stdout so server logs become indistinguishable from console text; and it gives up the
pmgui-gated queue’s clean line framing. Keep it in the back pocket only if a future PyMOL build
must run with -c.
8. Behavioural details the web client must handle
All measured ine11_edge.py / e12_long.py / e13_classes.py.
- Synchronous availability. 500 iterations of
cmd.do(...)+ immediate_get_feedback():0/500 drains missed their own line. On a single thread, feedback for a command is complete by the timecmd.doreturns. No settling delay needed. - Cross-thread polling works. A poller thread drained 407 lines including all 200
T-*lines while the main thread ranfragment/show surface/ray; 1Nonereturn (e6_thread.py). - Direct API calls are quiet;
cmd.dois not.cmd.fragment('ala')produced[], whilecmd.do('fragment ala')produced' Executive: object "ala" created.'. Route user-typed commands throughcmd.do()— that is where console parity lives. Programmatic calls made by the React UI (button clicks) should call the typed API and will correctly stay silent, or usecmd.doif you want them echoed like the Qt command line does. echo=0(cmd.do(x, echo=0)) suppresses thePyMOL>echo line but keeps the output.- Long lines are hard-split at ~1018 chars regardless of
wrap_output(defaultoff);OrthoLineLengthfail-safe atpackages/engine/layer1/Ortho.cpp:1097-1104. A 20000-char write became 20 feedback entries totalling exactly 20000 chars — no loss, but no 1:1 line mapping. Do not assume one feedback entry == one logical line for huge output. - Partial writes are buffered until a newline.
sys.stdout.write("PARTIAL")yields[]; the subsequentwrite("-CONTINUED\n")yields['PARTIAL-CONTINUED']. - stderr is captured too —
pcatch._install()setssys.stderr = sys.stdout = pcatch. - Full multi-line tracebacks come through, one feedback entry per traceback line, including the
Python 3.13
~~~~^^^^caret lines (e6.out). - UTF-8 is fine:
['PyMOL>print("ångström Å 你好")', 'ångström Å 你好']. - Progress bars are NOT in the feedback stream.
pmg_qtpollscmd.get_progress()separately (pymol_qt_gui.py:942→update_progress).cmd.get_progress()returns-1.0when idle. The web client needs its own progress channel. cmd.log_open()is not a capture route. Verified contents for a.pmllog:fragment ala\ncount_atoms all\nprint("logged?")— input echo only, zero output..pylogs givecmd.do('''fragment gly'''). Useful for session replay, useless for the console pane.
Observed line prefixes for client-side classification (e13.out)
Note the caret continuation lines (
"( ( ( (<--", "nonexistent_object<--") that follow selection
errors — they must be rendered with the preceding error line to be intelligible.
9. Deliverable — runnable capture module
Verified working end-to-end (§10). Drop this in the bridge package (owner’s choice of path; this spike creates no code files).Wiring notes for the bridge owner
FeedbackCapturemust be the only thing in the process that callscmd._get_feedback().- Serve the console over SSE/WebSocket by looping
wait_for(last_seq)and emitting the returned entries; the client sends back the highestseqit has, so a page reload replays cleanly from the ring buffer. - Use
fc.server_stdoutfor uvicorn/loggingoutput, or configure logging handlers beforefc.start(); anything written tosys.stdoutafterstart()lands in the PyMOL console pane. fc.stop()restoressys.stdoutbeforepymol.stop(). Reversing that order makes all subsequent prints disappear.
10. Verification transcript of the deliverable
test_feedback_capture.py, exit 0:
11. Required changes to other owners’ documents
These are reported, not applied — the files belong to other agents.docs/spikes/build.mdanddocs/build-and-tooling.md— the canonical smoke testpymol.finish_launching(['pymol','-cq'])and any bridge startup using-c/-cqmust not be carried into the bridge.-cpermanently disables console feedback. Keep-cqfor CI smoke tests only.docs/spikes/build.md§5.1 and §5.3 (this item used to cite an “action item (3)” that does not exist in that file) — “must usepymol2.PyMOL(), notpymol.finish_launching()” is half right. It must bepymol2.SingletonPyMOL().pymol2.PyMOL()breakspcatchand silently destroys all Python console output.docs/architecture.md— the bridge must setno_gui=0, internal_gui=0, internal_feedback=0, external_gui=0onpymol.invocation.optionsbeforestart(). Setting them afterwards has no effect (options are copied intoCPyMOLOptionsat_cmd._new).docs/architecture.md/internal-gui.md— the console feature rows should record that_get_feedback()is a destructive single-consumer read and that scrollback, sequence numbers and replay are the bridge’s responsibility, not PyMOL’s.docs/internal-gui.md— progress reporting is a separate channel (cmd.get_progress()), not part of the feedback stream.- Headless feedback capture — the blocker this spike was written to answer can be marked resolved; no C++ change required.