Parley: User Studies for Syntax Design

Tags: April 1, Pyret

Posted on 01 April 2014.

Programming languages' syntax choices have always been cause for spirited, yet unresolvable, debate. Opinions and so-called best practices dominate discussion, with little empirical evidence to justify them. As part of the Pyret project, we've been performing one of the most comprehensive empirical studies on programming language syntax.

To recap some of the issues: Many languages repurpose plain English words for keywords, and run afoul of the impedance mismatch between, for instance, the dictionary meaning of switch and its semantics within the language (see Language-Independent Conceptual "Bugs" in Novice Programming for a much more detailed discussion). Another alternative, using non-ASCII symbols which cannot have their meaning conflated, is promising but doesn't work well with traditional editors (APL, we're looking at you).

We are, instead, evalauting the use of unconventional syntax motivated by a non-technical, easily-interpreted lexicon. We refer to these cohesive lexicons as lingos, and have begun experimenting with one lingo-based syntax for Pyret, which we call Parley. It is best to see the Parley lingo in action, compared to the more traditional syntax in early versions of Pyret:

var sum = 0
var arr = [1,2,3,4,5,6,7,8]
for each(piece from arr):
  sum := sum + piece
end

yar sum be 0
yar arr be [1,2,3,4,5,6,7,8]
fer each(piece of arr):
  sum now be sum + piece
end

While it should seem obvious to the casual reader that Parley lingo is a strict improvement, this is not a substitute for empirical evaluation. To this end, we have been running a rummy series of user studies to validate the new syntax. Our latest experiments are testing program comprehension using an aye-tracker. The results, which will be submitted to the Principles of Pirate Lingo conference, are pending clearance from our Aye Arr Bee.