Whoosh

Full text search library

As part of Houdini’s help server system, I wrote a full-text indexing and search library in pure Python called Whoosh. Over the years I added more and more features and optimized to the point where it has all the features (and more) of widely-known projects like Lucene, but is just slower because of Python.

In 2013 I presented a talk about Whoosh at PyCon, the worldwide Python conference.

Whoosh now has a centralized community fork on Github, though I am still the main developer.

• Feature-complete (matches or exceeds the features of any other search libraries).

• Up to date with Information Retrieval research (up to the point when ML took over).

• Incremental re-indexing, custom search clauses, programmable query parser, hierarchical searching, and much more.

• Every part is replace-able with custom Python code.

• Very fast (for Python code). Uses mmap, zero-copy, and other low-level optimizations to improve performance.

• PyCon talk

• GitHub

Houdini help server

Wiki parsing and serving platform

Houdini is an extremely powerful application with many interfaces to support different jobs in a film production pipeline, such as modellers, character riggers, animators, lighters, and compositors. It’s also based on nodes, and every node needs its own documentation of what it does and how its parameters affect how it works.

This means the amount of documentation required is enormous. The kind of simple static generation tools widely used by other projects, like Sphinx, simply can’t scale to this size. And heavy-duty XML-based solutions like DITA are overly complex and don’t mesh well with developer workflows.

For all these reasons, I developed a proprietary wiki-based help system which can handle the extreme requirements of Houdini’s documentation set.

• Constantly running server and wiki parser let developers/users write docs in a text editor and see the preview update in the browser.

• Since the help server is running on the same Python interpreter used for scripting Houdini, the docs can pull data from the running Houdini instance.

• User-created content (such as custom nodes) automatically become part of the help presented to the user, and are searchable thanks to Whoosh indexing in the background.

• Wiki parser parsed to a JSON intermediate form, which is then rendered into HTML using Jinja templates. This system is extremely powerful and flexible.

Houdini info window

Inspector UI in Python/Qt

Originally, this inspection window was displayed using white-on-black, monospaced text, produced from C++ code much like printing to a console.

I used PySide to prototype and implement a new info window based on Qt, using colors, graphics, icons, and spacing to try to make the information clearer and easier to understand at a glance.

Node search

A little feature in Houdini that helps a lot

One day I noticed that the user interface for finding a particular node, according to very limited search criteria, was terrible in Houdini. So, I decided to do something about it.

I prototyped and implemented a new “Find Nodes” dialog with a “quick search” box at the top and a “match all/any of the following…” interface to build more complex queries based on a wide set of available criteria.

This slowly became a very popular feature as people discovered it, and requested more and more different search criteria for their individual workflows.