Arti: Welcome

Welcome to Arti!

The Associate Railroad Track Inspector, under administrative direction, conducts complex investigative and surveillance activities ...
– job description, California Public Utilities Commission

Arti is an attempt to improve the documentation infrastructure for Ruby in general and Rails in particular. Large numbers of Rails sites are under development. As these come online and go into maintenance, their developers and owners will find themselves needing better tools than are currently available.

Arti is an exploratory project in this direction, based on both existing and new technology. Although it is currently limited to static analysis techniques, dynamic techniques are on the agenda. Thus, Arti will take advantage of both the convention-based organization of Rails and the strong introspective features of Ruby.

Background

All these relationships can be confusing.
– Agile Web Development with Rails, 2nd. ed.

Like other dynamic programming languages, Ruby is not an easy target for documentation generators. Almost any part of the program can be (re)defined at run time, so static code analysis may fail to ascertain many aspects of the program. Systems such as Rails, which make extensive and intricate use of Ruby's dynamic capabilities, are particularly hard to analyze.

RDoc, designed by Dave Thomas, is the de facto standard for Ruby documentation generators. It analyzes Ruby source code and collects embedded comments, generating a structured collection of pages for Ruby files, objects, and methods. RDoc is included as part of the Ruby core distribution; commands to use it (e.g., "rake doc:app") are built into Rails.

Nonetheless, the RDoc support in Rails is not very extensive, let alone well integrated into the toolkit. Only the application code is analyzed; libraries (whether for Rails or other Ruby topics) are not documented. Also, RDoc takes no advantage of the extremely regular structure and naming conventions of Rails. So, it knows nothing about actions, environments, controllers, helpers, layouts, models, partials, tests,views, and so forth.

The Ruby support could also use improvement. Although syntax coloring is provided for the source code snippets, no support is provided for linking from (say) method calls to their definitions. Nor, by default, can the full source code of the application be perused through the RDoc interface.

Finally, because RDoc's design and implementation predate Rails, it generates static web pages that cannot be browsed through Rails, let alone take advantage of Rails' dynamic capabilities. Although RDoc has taken us a long way, it is clearly time to put in some more work on the problem.

Approach

Three great virtues of programming are laziness, impatience, and hubris.
– Larry Wall, creator of Perl

By means of some (admittedly kludgy) static analysis code, Arti is beginning to take advantage of the pervasive Rails conventions (see Rules). This allows it to print an annotated file list, some simple statistics and overview diagrams (see below). Arti also makes the full complement of the application's text files and RDoc pages available for viewing.

Arti can work in either "merged" or "remote" mode. In merged mode, Arti's code and data are commingled with those of the target application. In remote mode, Arti runs as a separate Rails app, minimizing its impact on the target application. See the install notes for more information.

Overview Diagrams

Using Graphviz (specifically, dot), Arti generates two types of image-mapped (i.e., clickable) overview diagrams:

File usage, by View
These diagrams show the Controller/Model and View phases, detailing the files that are used by each. A set of "hint" files (one for each diagram) allow the user to specify details that Arti cannot (yet) determine.
View Linkage Summary
These diagrams show the relationship between different Views. At present, all of the supporting information is entered by hand, but I hope and expect to remedy this situation as time goes on.

Science Fiction

Arti's feature set is still under rapid expansion. Here are several possible additions that are under consideration:

The "annotate_models" plugin promises to document models (and underlying databases).
Insurance is a test coverage analysis tool for Ruby.
model-graph generates diagrams for ActiveRecord models.
The Rails Application Visualizer generates diagrams for both controllers and models.
RailStat is a real-time web site statistics package.
Visualize Models generates diagrams for databases and models.
YARD (Yay! A Ruby Doctool) is an extensible tool for gathering information.

I have also been working on generating detailed execution logs, using Ruby's "hook" and other callback facilities. This works well on simple Ruby programs, but it needs work in order to handle the vagaries of Rails.

Once the logging code is able to reliably document the operation of a Rails app (e.g., while generating a page), I plan to start exercising apps and analyzing the results to elicit program flow, etc.

I plan to store this information, along with metadata Arti derives from the file tree, in a set of Rails-friendly database tables. I will then experiment with navigation and presentation ideas, including some notions for dynamic, GUI-based interaction. The high-level system design will be guided by my notions on Model-based Documentation.

Distribution

Although these are very early days in the Arti project, I believe that it is already showing substantial progress. So, I am making the source code available for folks to install and try out. Although I cannot provide unlimited free support for Arti, I'm quite willing to answer simple questions, handle bug reports and other feedback, etc. So, please feel free to send me a note.