HereDoc : "Here Document" facility for OCaml

release 2001-07-13

Introduction
Installation
The module Text: Convenient text manipulation
Quotation
Verbatim includes and templates

1. Introduction

This file doc.sections.tpl, last modified on 2001-07-14.

Without any doubt, OCaml is a great langage for Web publishing. I used it for a long time to produce my static web pages and more recently to write some CGI scripts.

HereDoc is an attempt to provide syntactic sugar for such applications.

In OCaml, strings constants can span multiple lines. Of course, this is most useful for the applications we have in mind. But OCaml lacks some imporant features.

you have to escape every " (the string delimiter in OCaml) in a string; for instance, the produce the string <a href=".."> you have to write <a href="..."> and, believe me, it becomes quickly annoying.
if you want to insert the content of an identifier or the result of a Caml expression (computed value), you have to do things like :
- ```
print_string ("The value of x is " ^ x ^ ".");
```
- ```
Printf.printf "The value of x is %s." x;
```
In Perl and similar languages, you just write :
- ```
print "The value of x is $x.";
```
which looks nicer.
there is no easy way to include file, at compile time. We would like to include files verbatim or use them as templates (text with gaps to be filled).

HereDoc relies on Camlp4, the powerful preprocessor for OCaml, to extend the concrete syntax of the language and provide convenient notations for all these operations. Most of the work is done at compile time, so the code is fast.

2. Installation

HereDoc has been developped on a Linux box, with:

OCaml 3.00
Camlp4 3.00

To install HereDoc, just untar the distribution tarbal, and do:

make HereDoc

This will produce the files:

pa_HereDoc.cma, text.cmo, text.cmx

To compile a file with the HereDoc syntax, you must pass to the compiler the option:

-pp 'camlp4o pa_HereDoc.cmo'

(put the correct path to pah_HereDoc.cmo).

To link, you'll probably need text.cmo (for the bytecode compiler) or text.cmx (for the native compiler).

To gain a better understanding of HereDoc, you may want to see the plain OCaml code it produces. First, you should build a version of HereDoc producing source code instead of an abstract syntax tree representation :

mkcamlp4 -o HereDoc_dump str.cma pa_o.cmo pr_o.cmo hereDoc_lexer.cmo pa_HereDoc.cmo

or simply:

make HereDoc_dump

Then, for instance:

./HereDoc_dump yourfile.ml

CamlP4 can't print everything, so the result of HereDoc_dump may not be usable as an input for OCaml.

Have a look at the Makefile and the files doc.ml, doc.layout.tpl, doc.sections.tpl (the source code for this page). They illustrate the use of HereDoc.

3. The module Text: Convenient text manipulation

Before we describe the new syntax, we have to introduce a tiny module : Text (what an original name). With this module, every Caml value "is" a string. To understand what it means, let's define the "string content" S(x) of a value x. It is a string, defined by:

if x is a string, S(x)=x
if x is an int, a char, or a constant constructor of a concrete type, S(x)=""
if x is a block {x1,...,xn}, S(x) is the concatenation S(x1)^...^S(xn)

Similarly, the "string components" SC(x) is defined by:

if x is a string, SC(x)=[x]
if x is an int, a char, or a constant constructor of a concrete type, SC(x)=[]
if x is a block {x1,...,xn}, SC(x) is SC(x1)@...@SC(xn)

The module defines an opaque type Text.t with an "universal constructor" Text.repr: 'a -> t. The function Text.iter:(string -> unit) -> t -> unit applies a given function to the string components of a Text.t value. There are also two functions to compute S(x) and its length.

So, for instance, we have (oh yes, this is not well-typed; you have to add Text.repr everywhere):

```
S("Hello world !") = "Hello world !"
```

S(["Hello";(" ","World");Some " !"]) = "Hello world !"

S([|"Hello"; 5; Some (5," "); "World !"|]) = "Hello world !"

Typically, an application builds its output by concatenating strings. This involves many copy operations so it may be quite slow. With the module Text, just put your substrings in a list (or better, an array: only one big block), and see it as a string with Text.repr. When you want to finally output the "string", use Text.iter or Text.to_string on the result.

Another feature of the module Text is the manipulation of "postponed texts". Suppose you want to create a text with a hole to be filled later, when an extra piece of information is available. A simple way to use a reference:

   let post = ref Text.empty in
   repr [Text.repr "Total size :"; Text.repr post];
   (* ... *)
   post := Text.repr (string_of_int (total_size ()));

Sometimes it is better to put the expression to be computed later at the position of the postponed text, then to activate it later; also, having a global reference is not easy to manage. You can do that:

   repr [Text.repr "Total size :"; Text.postponed "totalsize" (fun ()
-> Text.repr (string_of_int (total_size ())))];
   (* ... *)
   Text.activate "totalsize"

Here, "totalsize" identifies the delayed computation and Text.activate triggers the evaluation.

4. Quotation

HereDoc defines the quotation <:here<...>>. It returns a value of type Text.t (see the Text module). HereDoc declare this quotation as the default quotation, so you can simply write <<...>>.

The quotation content is interpreted with the following rules:

Anti-quoted expression:
The sequence $$ marks the beginning and the end of an expression to be evaluated inside the quotation. For instance:
```
 <<The winner is $$[|"John";"Bob"|].(win)$$. >> 
```
You can use any identifier in the scope at the quotation position. The expression can return a value of any type, and it will be interpreted with the module Text (with Text.repr). For instance, it can return a list of strings, and it will be interpreted as you would expect:
```
 <<The winners are : $$List.map String.uppercase ["John";"Bob"]$$>> 
```
Note that inside the expression, you can use another quotation (yet another stupid example):
```
 << Tada $$ (<<Hello>>, " World") $$ >> 
```
Anti-quoted identifier:
There is a shorter form when the expression is only a identifier, possibly qualified with a module path and/or a record path:
```
 <<And the winner is ... $winner !>> 
```
```
 <<And the winner is ... $Game.winner !>> 
```
Runtime variable:
The $$ form is expanded at compile time, so the name of identifiers must be known to the compiler. You can't access directly "run-time" variables, like CGI arguments. Suppose you have a function
```
val var : string -> 'a = <fun>
```
which can expand these run-time variables (for instance, it can lookup in the table of CGI arguments) where, as usual, the result is to be interpreted through the module Text. You can write:
```
 << $$var "street"$$>> 
```
You have a short form for this:
```
 << $(street)>> 
```
The name of the expanding function is "var" by default; but you can change this with the toplevel directive:
```
VAR_HANDLER "myfun"
```
It is important to understand that the name of the expanding function is static, but the real function depends on the scope of the quotation. Here's the traditionnaly stupid example:
```
    let var = function
       | "birthday" -> "May 13th"
       | "firstname -> "Alain"
    in	
    << This year, $(firstname)'s birthday is on $(birthday). >>
 
```
Conditional sections:
you can define conditional sections with the notation:
```
 $?expr: 
```
where expr is a boolean expression. If the expression evaluates to true (during runtime), the text between this conditional and the next one (or the end of the quotation) is included. Otherwise, it is discarded.
The empty conditional $?: is equivalent to $?true:.
Here is an example:
```
       let (fr,en) = (true,false) in
       let name = "John" in
       << $?fr:Bonjour$?en:Hello$?: $name >>
     
```
You can use more complex expressions:
```
       << $?Random.int 10 = 0 :lucky man$?: >>
     
```

Abstraction:

Consider the following example:

	<ul>
	$$List.map (fun x -> << <li>$x</li> 
	>>) l$$
	</ul>

HereDoc provides a nicer notation for this kind of iteration:

       $[e -> p1 -> p2 ... -> pn]$
       ....
       $[]$

is equivalent to:

      $$e (fun p1 ... pn -> <<....>>)$$

e is an expression which evaluates to a function with n curryfied arguments. p1, ..., pn are patterns. The last .... is called the abstracted text. For instance, suppose you have defined:

        let map l f= List.map f l
        let rec mapi n l f = 
          match l with
            | [] -> []
            | t::q -> (f n t)::(mapi (n+1) q f)

You can write this html table renderer:

       <table>
       $[mapi 1 t -> i -> ligne]$
       <tr>
       <td><b>$$string_of_int i$$:</b></td>
       $[map ligne -> case]$
       <td>$case</td>
       $[]$
       </tr>
       $[]$
       </table>

The case n=0 is useful to filter the text through a function. For instance, if box: Text.t -> Text.t is a function which put a frame around its argument, you can write:

       $[box]$ Long long text ...  $[]$

Note: Conditional sections are local to the abstracted text.

Specials:

You have a few metavariables accessible with the notation ${meta} or ${meta:arg}. Here are the possible values for meta:

filename Returns the current file being processed (the ml source file or template file, see later); arg is ignored.

lastmod Computes the last modification date for the current file; if arg is empty or absent, the date is formatted as "yyyy-mm-dd"; otherwise, arg is parsed as a Caml expression yielding a formatting function int -> int -> int -> 'a (whose arguments are in the order year, month, day).

You can of course escape (protect) $$ and $ by a backslash: \$$ \$. The backslah itself can be escaped: \\.
A backslash at the end of the line hide the following \n character. All other backslash sequence, such as \n, \t, etc... are disabled. You don't need to escape the quotation mark.

5. Verbatim includes and templates

It is sometimes convenient to put some text in external file. HereDoc extends the syntax to provides four kind of includes: verbatim, templates, declaration includes and expression includes.

VERBATIM "file.txt" returns a string with the content of the given file.
TPL "file.txt" returns the content of the file, interpreted as a quotation (of type Text.t).
INCLUDE "file.txt" (at the position of a toplevel declaration or of a structure implementation item) includes the content of the file.
EXPR "file.txt" same as INCLUDE, but for an expression.

To print a template, use:

Text.iter print_string TPL "file.txt"

You can put several chunks of text in a file. Use the notation:

"file.txt"."chunk"

to designate the section "chunk" in the file "file.txt", that is all the lines between the line "==chunk==" (two signs equal, the name of the chunk, two signs equal, alone on their line), and the line "====" (four signs equal). Templates and expression includes may take arguments: if the opening ligne is ==chunk arg1 arg2 .. argn== instead of ==chunk==, the chunk returns a function with labeled arguments arg1,...,argn. For instance, if the template file is:

 ==add x y==
 x + y
 ====

 ==link dest txt==
 <a href="$dest">$txt</a>
 ====

you can declare:

let add = EXPR "file.txt"."add"
let link = TPL "file.txt"."link"

and get:

val add : x:int -> y:int -> int
val link : dest:'a -> txt:'b -> Text.t

If many of the chunks you need are in the same file, you can use the toplevel phrase:

TEMPLATE_FILE "file.txt"

and then omit the file name, for instance:

TPL ."chunck"

When parsing an external template, the default file is the current file. By default, the template filename is "templates.tpl".

The commands "VERBATIM", "EXPR" and "TPL" are expressions; you can use them inside "expression in quotation". So you can for instance include a template from a template :

 ==chunk==
 Want to see another template ?
 $$TPL ."another"$$
 ====

Return to the top

Author's mail : Alain.Frisch@ens.fr
Author homepage
The program took 0.01 seconds to generate this page.

filename	Returns the current file being processed (the ml source file or template file, see later); arg is ignored.
lastmod	Computes the last modification date for the current file; if arg is empty or absent, the date is formatted as "yyyy-mm-dd"; otherwise, arg is parsed as a Caml expression yielding a formatting function `int -> int -> int -> 'a` (whose arguments are in the order year, month, day).