http://www.perlmonks.org?node_id=553487

The most important aspect of any module is not how it implements the facilities it provides, but the way in which it provides those facilities in the first place.

-- Damian Conway in Ten Essential Development Practices

Public APIs are forever -- (you have) one chance to get it right.

-- Joshua Bloch in How to Design a Good API and Why it Matters

Interfaces matter. The more public they are, the more they matter. Indeed, once a public API -- such as Conway's CPAN modules or Bloch's Java SPIs -- attracts widespread usage, changing it becomes practically impossible. By contrast, just about anything else can be easily fixed in a later release.

And yet -- as indicated by Brooks' famous aphorism, plan to throw one away; you will, anyhow -- you're most unlikely to concoct the perfect API at your first attempt.

Making the task more daunting still, the dark art of interface and API design is certainly not an easy one to master, there being many disciplines in play: computer science, human factors (ergonomics), cognitive science, psychology, sociology, linguistics, usability, and so on.

I've spent the past few weeks researching this difficult topic and found very few references dedicated to this subject. Instead, many books devote a section or two to the art of interface design, or perhaps mention it in passing while analyzing a knotty design or coding matter.

This meditation reports the interface and API design references I've found useful and further presents some general interface design ideas and checklists in the hope that they may prove useful -- and that they might be improved upon by your insightful feedback.

Bloch's Seven Characteristics of a Good API

In his How to Design a Good API and Why it Matters keynote, Joshua Bloch proposed seven characteristics of a good API:

  • Easy to learn.
  • Easy to use, even without documentation.
  • Hard to misuse.
  • Easy to read and maintain code that uses it.
  • Sufficiently powerful to satisfy requirements.
  • Easy to extend.
  • Appropriate to audience.

Bloch also cautions that you need at least three different implementations of an API before you can be confident of its soundness.

Conway's Sufficiently Advanced Technologies (S.A.T)

In module design, interface is everything. Going one step beyond this dictum, Damian demonstrates and explains several practical applications of Clarke's Law ("Any sufficiently advanced technology is indistinguishable from magic") by presenting a series of useful modules whose interface is...nothing

-- from advertising of Damian Conway's S.A.T seminar

Like Bloch, Conway proposed seven API design tips, namely:

  • Do one thing really well.
  • Design by coding.
  • Evolve by subtraction.
  • Declarative trumps imperative.
  • Preserve the metadata.
  • Leverage the familiar.
  • The best code is no code at all.

Some example S.A.T.-esque modules that achieve a lot with very little interface are: strict, diagnostics, Smart::Comments, Perl6::Say, Perl6::Slurp, IO::Prompt, Getopt::Clade, Getopt::Euclid, Tie::File, Win32::TieRegistry, IO::All.

Further tips gleaned from listening to TheDamian's YAPC::Asia 2006 S.A.T talk are:

  • Evolve your module's interface by examining its clunky, awkward bits and eliminating them.
  • To evolve by subtraction, ask: "what can be removed?", "what can be automated?", "what can be (contextually) inferred?".
  • Make common usage the default; allow uncommon usage via optional attributes.
  • Two of Perl's most endearing qualities are: it does the hard work for you; it doesn't get in your way. Make those the strengths of your module too. For example, a module with too many methods (aka a fat interface) gets in your way.
  • The main idea behind all this is to support coding with no conscious effort.

Perl 6 Design Principles

The linguistic and cognitive principles behind the Perl 6 design are nicely laid out in Perl 6 and Parrot Essentials as follows:

  • The waterbed theory of complexity. Languages tend to have similar overall complexity.
  • The principle of simplicity. Prefer simple to complex. Beware of false simplicity.
  • Huffman coding. Reserve the best shortcuts for commonly used features.
  • The principle of adaptability. Design for change and growth.
  • The principle of prominence. Mark prominence with stylistic devices.
  • The principle of end weight. Place large complex elements at end of sentences.
  • The principle of context. Use context to interpret meaning.
  • The principle of DWIM. Use programmer's intuitions, don't fight them.
  • The principle of reuse. Reuse the same structures in different contexts.
  • The principle of distinction. Different things should look different. Distinction and reuse are in constant tension.
  • Language cannot be separated from culture.
  • The principle of freedom. TMTOWTDI.
  • The principle of borrowing. Borrowing is common in natural and computer languages.

API Design Checklist

After all that, here's my attempt at an API design checklist:

  • Make it easy to use correctly, hard to use incorrectly. To illustrate, Scott Meyers gives a cute example of a Date class with constructor Date(int month, int day, int year), making it easy to use incorrectly (i.e. to get the parameter order wrong). Another illustration is the PBP guideline: "Use a hash of named arguments for any subroutine that has more than three parameters", which works well because humans are better at remembering names than orderings.
  • "Play test" your API from different perspectives: newbie user, expert user, maintenance programmer, support analyst, tester. In the early stages, imagine the perfect interface without worrying about implementation constraints. Design iteratively. See also: on TDD.
  • Names matter. Choose names that are explanatory, consistent, regular.
  • Have guiding principles and clear requirements.
  • Define sound conceptual models and domain abstractions.
  • Consider whether your domain functionality is best delivered as a library (e.g. DBI), an application framework (e.g. Catalyst), or a DSL (e.g. YACC (a translator) or make (an interpreter)).
  • Hide implementation details. Reflect the user mental model, not the implementation model. Information hiding: Minimize the exposure of implementation details; provide stable interfaces to protect the remainder of the program from the details of the implementation (which are likely to change). Don't just provide full access to the data used in the implementation.
  • Make easy things easy, hard things possible. Huffmanize.
  • For non-public APIs, be sufficient, not complete. It is much easier to add a new feature than to remove a mis-feature. When in doubt, leave it out.
  • When in doubt, or when the choice is arbitrary, follow the common standard practice or idiom.
  • Consider the API from the perspectives of: usability, simplicity, declarativeness, expressiveness, regularity, learnability, extensibility, customizability, testability, supportability, portability, efficiency, scalability, maintainability, interoperability, robustness, type safety, thread-safety/reentrancy, exception-safety, security. Resolve any conflicts between perspectives based on requirements.
  • Error handling. Document all errors in the user's dialect. Prefer throwing exceptions to returning special values. Prefer to find errors at compile time rather than run time (e.g. PBP p.182, Named Arguments: pass as a single hash ref not a list of name/value pairs, so that some argument errors are caught at compile time).
  • Apply the Principle of least astonishment. In particular, choose sensible (expected) and secure defaults.
  • Look for ways to eliminate ungainly parts of the API (e.g. the need for repeated boilerplate code when using the API).
  • Plan to evolve the API over time.
  • Learn from prior art. In particular, avoid repeating interface design mistakes of the past.

Some Examples of Interface Mistakes

For some light relief from all those checklists, from the thousands of interface and API goofs that have been made over the years, I list here a few random ones I remember.

Perl 5's "string eval" and "block eval" is an example of violation of the "principle of distinction", aka "different things should look different" (this has been fixed in Perl 6, where block eval is now spelled try). This example illustrates the importance of choosing good names. Certainly, I've been shocked many times over the years at meeting experienced Perl programmers who were completely unaware that Perl supported exception handling and I feel that would not have happened had Perl sported a try keyword. (Update: Try Catch Exception Handling finally made it into core perl 5.34 as an experimental feature - see also New built-in perl5.38 try/catch syntax and SO question about Perl exception handling).

The Unix make utility decrees that the actions of a rule must start with a tab. If you accidentally insert a space before the leading tab, look out! Ditto if your editor or other tool is configured to automatically convert tabs to spaces. This unfortunate design choice is a violation of the principle of least astonishment because most programs treat spaces and tabs the same way. Moreover, when you hit a (typically cryptic) error message for using a space instead of a tab, it may take a long time to figure out what the problem is because tabs and spaces look the same in most editors.

Lexical file handles are, for me, the most important feature introduced in Perl 5.6. Those evil old global file handles spawned a host of unfortunate idioms, such as:

select((select($fh), $|=1)[0]);
In terms of a clear and simple interface to perform a simple task (set autoflush on a file handle), it doesn't get much worse than that, a compelling illustration of why keeping state in global variables is just plain evil. I won't dissect this horror further, other than to exhort you to replace it with:
use IO::Handle; # update: not needed for Perl 5.14+ # ... $fh->autoflush();

Another file handle annoyance is:

print $file $name, $rank, $serial_num, "\n";
Do you want $file to be treated as a filehandle ... or did you just forget a comma? Two ways to clarify are:
print {$file} $name, $rank, $serial_num, "\n";
and:
use IO::Handle; $file->print( $name, $rank, $serial_num, "\n" );

To round out this section, notice that the ANSI C strtok function has a shocker of an interface: it surprises by writing NULLs into the input string being tokenized; the first call has different semantics to subsequent calls (distinguished by special-case logic on the value of its first parameter); and it stores state between calls so that only one sequence of calls can be active at a time (bad enough in a single-threaded environment, but so intolerable in multi-threaded and signal handler (reentrant) environments that the POSIX threads committee invented a replacement strtok_r function).

Human Aspects

The three greatest experts in the human side of interface design that I'm aware of are:

From The Design of Everyday Things, Norman's seven principles of design are:

  • Use both knowledge in the world and knowledge in the head.
  • Simplify the structure of tasks.
  • Make things visible: bridge the gulfs of Execution and Evaluation.
  • Get the mappings right.
  • Exploit the power of constraints, both natural and artificial.
  • Design for error.
  • When all else fails, standardize.

Some other principles derived from Norman and Nielsen's works are:

  • Affordances. It's vital to ensure the perceived properties of an object match its actual properties. Norman gives a lovely example of a sliding door with a conventional door handle that looks like you should press it down to open, when in fact you must slide it.
  • Things that are different should look different. Norman gives an amusing example of two similar switches on an airplane: one adjusted the wing flaps, the other lifted the wheels. As you might expect, there were a number of planes wrecked by pilots inadvertently lifting the wheels while taxiing down the runway.
  • Provide a good conceptual model.
  • Make things visible.
  • Provide natural mappings. A classic example is the humble kitchen stove; with a natural mapping of burners to controls, no learning or remembering is required and there is no need for labels on the stove controls.
  • Provide feedback.
  • Leverage the power of constraints.
  • Make it easy to remember.
  • Prevent, detect and forgive errors.
  • Make dangerous operations difficult to use. For example, child-safe gates to swimming pools, child-proof tops on bottles of dangerous chemicals.
  • Usability before aesthetics.
  • Go with the existing standard unless you have a significant improvement; especially go with the existing standard when the choice is arbitrary. For example, screw threads tighten clockwise, the hot water faucet is on the left, clocks turn clockwise (the backwards clock notwithstanding).
  • Beware of creeping featurism.
  • Make it explorable and discoverable.
  • Make the computer invisible.

GUI Design Checklist

Here's a brief GUI design checklist. For more details, see the most excellent About Face 2.0 by Alan Cooper.

  • Have clear objectives and guiding principles.
  • Define personas; design to satisfy their goals.
  • Adopt the user's perspective. Involve users in design. Perform usability tests. Give the user control. Make it configurable. Design iteratively.
  • GUIs should reflect the user mental model, not the implementation model. Hide implementation details.
  • Communicate actions to user. Provide feedback. Anticipate errors. Forgive errors. Offer warnings.
  • Be clear and specific in what an option will achieve: for example, use verbs that indicate the action that will follow on a choice.
  • Cater for both novice and expert. For novice: easy-to-learn, discoverable, tips, help. For expert: efficiency, flexibility, shortcuts, customizability. Optimize for intermediates.
  • Fit the appearance and behavior of the UI to the environment/process/audience.
  • Follow basic design principles: contrast (obviousness), repetition (consistency), alignment (appearance), proximity (grouping), balance (stability).
  • Keep interfaces simple, natural, consistent, attractive. Try to limit to seven simultaneous concepts.
  • Use real-world metaphors.
  • Ask forgiveness, not permission. Make all actions reversible.
  • Eliminate excise.
  • Be polite; remember what the user entered last time.
  • Avoid dialog boxes as much as possible; don't use them to report normalcy.
  • Provide "wizards" for complex procedural tasks.

References

References Added Later

Related Perl Monk Nodes

Many references were added long after the original post was made. Updated 10-jun: Minor wording improvements, expansion of "Conway's S.A.T" (thanks drbean), "Human Aspects", "GUI Design Checklist", and "References" sections. Updated 30-jan-2011: Added modernperl Ugly Perl reference. Updated 20-Jan-2021: Added makefile example where tabs are different to spaces.

Replies are listed 'Best First'.
Re: On Interfaces and APIs
by xdg (Monsignor) on Jun 04, 2006 at 13:56 UTC

    ++ for an amazingly comprehensive reference list.

    However, in trying to absorb the 39 various bullet points of proposed guidelines on API design, I'm reminded of the Justice Stewart quote on pornography:

    "I shall not today attempt further to define the kinds of material I understand to be embraced . . . but I know it when I see it . . . "

    Of the various guidelines, I like Bloch's best because of the short and clear concepts they present. The others need to be parsed and understood and use several metaphors that impede clarity unless you already know what the authors are talking about.

    Part of the challenge in giving API guidelines is that API style is context dependent as well. Different language styles lead to different API styles and what works best for one language might not work best for another.

    For the reference section, I'd suggest adding things like Tufte's VDQI, as the principles he lays out for eliminating non-data ink have some nice parallels to API design -- particularly of the Conway style. On the other hand, sometimes, as with named parameters, clarity and extendibility means more ink (er, pixels), not less.

    Separately, one reason that I like test-driven development is that by writing unit tests first before writing code, I wind up "using" my API before I implement it. At that point, the cost of change for the API is minimal. Even as code is implemented and additional features are added, the cost of change at that point is still pretty low. I'll frequently do several drafts of the API just while writing the test file.

    -xdg

    Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.

Re: On Interfaces and APIs
by jhourcle (Prior) on Jun 04, 2006 at 22:40 UTC

    Although many of Jakob Nielsen's observations are sound, I'm not a fan of his recent writing -- I stopped reading AlertBox a few years ago when too many of his postings seem to be of the 'here's some cursory information -- buy the whitepaper if you want more'.

    If you want a quick, simple book on usability concepts for web design, take a look at Steve Krug's Don't Make Me Think. It's the first book I typically give people on usability, as it's a very usable book itself.

Re: On Interfaces and APIs
by ForgotPasswordAgain (Priest) on Jun 04, 2006 at 19:51 UTC
    Nice review. I've been thinking a little about interfaces too, recently, in particular the Mozilla embedding API, which I've tried to understand for the last...year or so. That has to be one of the worst ever, though I'm sure the developers think it's a shining example of...something.

    I think the ideas for interfaces you pointed out should be extended to all code, not just modules. One of the things that takes me the longest in writing even relatively one-off scripts is that I spend a lot of time worrying about how much sense the script makes for whoever will eventually maintain it.

Re: On Interfaces and APIs
by adrianh (Chancellor) on Jun 05, 2006 at 13:34 UTC
    This meditation reports the interface and API design references I've found useful and further presents some general interface design ideas and checklists in the hope that they may prove useful -- and that they might be improved upon by your insightful feedback.

    For me the two greatest tools for interface/API design I've come across are Test Driven Development and Refactoring. Growing effective interfaces/APIs over time, rather than designing them all up front, is a stupidly useful technique.

    API Design Checklist

    Alan Shalloway and Ron Jeffries' list of features for a simple design is one that resonates for me:

    1. Runs all the tests.
    2. Contains no duplication.
    3. Expresses all the ideas you want to express.
    4. Minimizes classes and methods.
    The three greatest experts in the human side of interface design that I'm aware of are: Donald Norman, Jakob Nielsen, Larry Wall

    I know lots of people who would argue with Nielsen's place on that list - especially with the absence of Alan Cooper!

    While Larry is a hugely talented programming language designer (in my opinion anyway :-) I'm not entirely convinced that this necessarily maps onto "human" interface design in general.

Re: On Interfaces and APIs
by Polonius (Friar) on Jun 04, 2006 at 19:21 UTC

    Great review! I've read some, but by no means all, of these references before, but I've never seen them pulled together in one place. They're not specific to Perl, and this article deserves a wider audience.

    Polonius
Re: On Interfaces and APIs
by florg (Friar) on Jun 07, 2006 at 07:50 UTC
Damian SAT talk archived
by drbean (Novice) on Jun 07, 2006 at 03:38 UTC