Domain-Specific Languages and Language-Oriented Programming

May 6, 2009

Language-Oriented Programming (LOP): Build a language that makes your problem easy to solve, then solve your problem in it.

Domain-Specific Language (DSL): A language created by LOP.

DSL is frequently defined loosely. Most introductions tend to throw out a half-hearted attempt (e.g. “Domain-specific languages (DSLs) are languages tailored to a specific application domain.”), then list examples of some DSLs. You’ll know when you see one.

The above is my attempt at a different tack. A DSL is the byproduct of a LOP process. It’s a definition based on the historical intention of its creator rather than on intrinsic properties of a given language.

A crisp definition is irrelevant, anyway. “We must know for sure if this thing is a DSL! The Zombies are coming!”

Many successful DSLs exist, such as CSS, parts of Ruby on Rails, regular expressions and most declarative languages.

Much work has gone into DSLs. There are hundreds of articles online, several books, several academic papers, some conferences, and now an emerging industry of Language Workbenches led by JetBrains and Intentional Software. There’s also a programming language designed specifically for extensibility called XL. Whether or not some cautious empiricist has validated the claims of DSL effectiveness, something is clearly there.

I had no idea what I was getting myself into when I set out to find validations of DSL effectiveness, actually.

Internal and External DSLs

Martin Fowler on DSLs

Martin Fowler distinguishes between external and internal DSLs. An external DSL is a standalone new language with an interpreter or compiler. An internal DSL is an implementation within an existing programming language.

I’ll go one step further than Fowler and propose a mechanism that is common to all internal DSLs: An internal DSL is a mapping from the abstractions provided by the host language to domain concepts. Internal DSLs are abstraction hijackings. When your classes are types of objects and functions are functions, you’ve got a library. When your classes are database tables and your functions are relationships between them, you’ve got a DSL.

Some languages are more suitable for the creation of internal DSLs because they provide ways for their abstractions to be hijacked. Wherever an abstraction-hijacking isn’t perfect, the DSL has rough edges.

I need to be careful. If someone considers a large library of macros to be a DSL, this would not be an abstraction hijacking. Instead it operates at a purely syntactic level.

DSL Effectiveness

For research on DSL effectiveness, the internal/external distinction probably does not matter. Like the wings of bats and birds, we don’t care how they came to be—just that they both fly.

Consider information theory. A channel with fewer bits going over it can convey more information by increasing the assumptions the sender and receiver make. If you know that A means Apples, B means Bananas, C means Cherries and D means Dates, “4A2B1kgC500gD” can be a shopping list.

DSLs work on the same principle. We can convey more information in a shorter file if the reader makes more assumptions. The reader assumes domain knowledge.

“When and How to Develop Domain-Specific Languages”

Claim of the day:  “They offer substantial gains in expressiveness and ease of use compared with general-purpose programming languages in their domain of application.”

On the evidence: “…the benefits of DSLs have often been observed in practice and are supported by quantitative results such as those reported in Herndon and Berzins [1988]; Batory et al. [1994]; Jones [1996]; Kieburtz et al. [1996]; and Gray and Karsai [2003], but their quantitative validation in general as well as in particular cases, is hard and an important open problem.”

How do they do it?

  • Domain-specific notations (beyond the operators allowed in a general programming language)
  • Domain-specific abstractions (beyond functions, objects and other such abstractions)
  • DSLs allow domain-specific analysis.

This particular paper also reviews existing literature on DSLs, lists several tools that support DSL development, and provides high-level patterns for DSL development.

“Notable Design Patterns for Domain-Specific Languages”

Diomidis Spinellis provides a list of DSL implementation patterns. He also lists some advantages and disadvantages of DSLs in general, citing a lack of broad support in the industry as a primary disadvantage.

“Language Oriented Programming: The Next Programming Paradigm”

Sergey Dmitriev, the founder of JetBrains and creator of IntelliJ IDEA, believes LOP is going to change programming in a big way. He supports the development of MPS (and its integration with IDEA) to help this happen.

“Language-Oriented Programming”

LOP is proposed as an alternative to bottom-up, top-down and other program development styles. There’s an extensive example in the development of a program transformation system called MetaWSL.

Beyond LEX and YACC: How to generate the whole compiler

W. M. Waite briefly documents a number of DSLs to simplify compiler construction.

To Sum Up

DSLs let you do more with concise notations. This makes it easy to develop faster. Many smart people think Language-Oriented Programming is the wave of the future, but it’s hard to measure DSL effectiveness in general.

Finally, I have no idea how to squeeze a thesis out of DSL research, though I’d love to.


3 Responses to “Domain-Specific Languages and Language-Oriented Programming”

  1. Greg Wilson Says:

    RQ1: Are programs written in DSLs easier or harder to debug than programs written in general-purpose languages?

  2. Edward Ocampo-Goodinh Says:

    I’d like to see this thesis happen.

    I think I’m also missing out on something given that I see any well-written program to always be written in a DSL; the degree of the layer hijacking changing with the difference in language needed to sufficiently solve the problem in a clean, and easily comprehended way.

  3. Ian Says:

    Interesting problem. I only have limited exposure to DSLs, but ff you accept that SQL is a DSL, then for a DSL like SQL, there is little to no debugging capability – you have to do the equivalent of adding prints in by temping results and dumping them out … messy … and it does not always work, because sometimes the intermediate results are not temp-able. Typically you debug the query by pulling it apart/building it up. I have not used any query generators, but maybe they provide a debugging-like capability.

    I don’t know if you would consider stored procedure languages (SQL/PL, etc), built on top of SQL to mimic more general purpose language capabilities, to be a DSL, but they sometimes do have debugger support.

    Another interesting question that comes to mind (and maybe has already been studied) is the benefits of an IDE for a DSL – does that make using the DSL even easier – especially in the case of a novice? That would be a fairly easy thing to study too, if you can find a DSL with and IDE that ‘normal’ people would have limited exposure to…

Comments are closed.

%d bloggers like this: