Introduction

This is not exactly a feature by itself (it is part of the Filter feature [core]), but I felt like there was a need for a specific page to talk about this feature.

The token replacement feature is a way to speak of the capability for the system to provide a set of variables that can be used as replacement of tokens found in pages.

There is one main reason for making this feature part of core: we want ONE scheme that works for all the plugins in a seamless manner.

Syntax

The syntax is as follow:

token ::= '[' name ( '(' parameters? [,]? ')' )? ']'

name ::= [_a-zA-Z] ( [:_a-zA-Z0-9]* [_a-zA-Z0-9] )?

parameters ::= value
             | name '=' value
             | parameters ',' parameters

value ::= (-|+)[0-9]('.'[0-9]+)?
        | '"' any '"'
        | '\'' any '\''
        | name

any ::= .

Parameters may be named or not. It is often that the name of the tag is viewed as the name of the first parameter (say you want to display the content of another page, you could use something like [page("flowers")] where flowers is a page name, it could thus have been written as [page(name = "flowers")] and got the same effect; although the name of nameless parameters is left undefined.)

The string syntax allows for pretty much any character to appear in the string (i.e. to include a closing square bracket, a semi-colon, or colon.) It still doesn't support new-line characters for which users could use &#xD; although it is very rarely needed in HTML and the opening quote cannot appear in the string, although if necessary &quot; or &apos; can be used. At this point we forbid HTML tags in tokens (especially the unescaped < and > characters.)

Some tokens do not require the parenthesis, for example [year] simply prints out the current year. Many tokens are defined by other plugins in which case the name will be qualified. For example, [users::picture] is used to display the picture of a user registered with a Snap! website.

Not shown in the syntax up there, you may include a sub-token within a token (fully recursive) list of parameters. This means you could use a dynamic value as the parameter of another token (TODO: make it a real example that works...):

[versions::plugin([images::name])]

This example would first replace [images::name] with an image name, say that is "happy", then the system will call [versions::plugin("happy")] and replace that with the corresponding version.

Implementation Details

We parse the name of the token and all of its parameters, then we send a signal to all the installed plugins. The first that knows about this token name replaces it with the expected result. At this point the interface makes use of a pityful object. We want to ameliorate the implementation in that regard (i.e. the structure used allows you access to the parameters and let you make modifications to the existing result if there is one.)

When a token cannot be transformed, the page is considered at least semi-invalid if not completely invalid. That should generate an event (especially if the backend finds such problems,) so the author can be asked to fix the problem. There can be two main reasons for transformation failure:

1) The token syntax is incorrect, and

2) No plug-in know about that token in the current context.

We also want a way to escape tags with a simple syntax such as \[this-is-not-a-tag] (notice the backslash introducer.)

Contexts

The context is determined by the data being parsed. It is not always a page per se.

The available tokens and parameters vary depending on the context. For example, you cannot print out the name of the page if you are outputting something else than a page.

The context is determined by the parameters offered to the token transformation capability. For example, the Snap! Websites offers a way to define a default URL using the title of a page, the parent page(s) of the current page, etc. When this process happens, the page and website objects are both passed along the request allowing all the variables available in those two objects to be accessed by the different plugins to generate the necessary data.

To continue on the title of a page and parent of a page, in a book, you could get a URL that makes use of the parent page URL and appends the current page title to that URL and use that new full path as the path to the new page. Without the book feature, access to the parent URL would not be availabe.

Similarly, the book plugin offers tokens that output a link to the Next Page, Previous Page, Top Page (home/index of the book), Index Page, Glossary Page, Author Page, Copyright Page, etc.