I have just learned about dogtail, and it can allow enumeration of menu items/widgets of a GUI application from the command line; for instance (via Testing/Automation/DogtailTutorial - Ubuntu Wiki):

%> python
>>> from dogtail import tree
>>> gcalctool = tree.root.application('gcalctool')
>>> gcacltool.dump()
# You'll see something like: 
{"gcalctool" application}
 Node roleName='frame' name='Calculator  - Scientific' description=''
  Node roleName='filler' name='' description=''
   Node roleName='menu bar' name='' description=''
    Node roleName='menu' name='Calculator' description='' text='Calculator'
     Node roleName='tear off menu item' name='' description='' text=''
     Node roleName='menu item' name='Quit' description='' text='Quit'
     Node roleName='menu item' name='Empty' description='' text='Empty'
    Node roleName='menu' name='Edit' description='' text='Edit'

However, it uses (as far as I can see) assistive/"accessibility technologies" of Gnome Desktop. So I was wondering - is there such a tool (which can at least enumerate widget names) which works on a lower level - e.g. on XWindows level?

I guess (but do not know) that XWindows doesn't in itself offer any standard widget framework - and thus, it would be difficult to write a tool to query the GUI widget names on X-Windows level; is this correct? (I imagine something like xdotool or wmctrl, but neither of these can probe "inside" the GUI app).

If so - is there a tool that can work with most desktop environment frameworks (Gnome, KDE ...) that could query/list the widget names of a running GUI app, but without using the accessibility technologies layer?

  • I think best bet would be to search not in X level, but in GTK and QT level. X window really in modern toolkits just is thing which provides windows as canvas to draw stuff on it, compositor is the real widget renderer, that is the reason why Wayland will replace X. Instead you should try to detect what toolkit on window is used (QT or GTK) and try to detect widgets in their provided methods. – IBr Oct 18 '13 at 6:50
  • I think you are looking for xwininfo. However, it will only show you windows. X only knows windows, not widgets. Some toolkit widgets have corresponting X windows, some don't. For toolkit specific information, you need tools for the individual toolkits. – Bananguin Oct 18 '13 at 12:36

Twenty years ago, the answer would have been editres. Classical X Window programs used the X toolkit, which provides a standard way to define a widget hierarchy and configure it via resources. Many programs that use the X toolkit implement the Editres protocol which allows applications to query another application's widget tree and resource values.

Modern widget toolkits such as Gtk and Qt do not support X resources, let alone the Editres protocol. So you're out of luck. Someone did propose Editres support in Gtk+ but the patch was never adopted, and an application would have to include additional support even if the library did have the requisite extensions. For Qt (i.e. KDE applications), this old post from a Qt mailing list suggests KD Executor, but I have no idea whether it can actually do this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.