Token nodes¶

For every input token, there is an associated token node. Token nodes are leaf nodes in the parse tree. Tokens always have a token symbol. At lexing time, they can be assigned a token value. If no token value is assigned at lex time, the value of the token is a Perl "undef".

Rule nodes¶

If a node is not a token node, then it is a rule node. Rule nodes are always associated with a rule. It a rule's action is a rule evaluation closure, it is called at Node Evaluation Time.

The rule evaluation closures's arguments will be a per-parse-tree variable followed, if the rule is not nulled, by the values of its child nodes in lexical order. If the rule is nulled, the child node values will be omitted. A rule evaluation closure action is always called in scalar context.

If the action is a constant, it becomes the value of the rule node. If the action is a rule evaluation closure, its return value becomes the value of the node. If there is no action for a rule node, the value of the rule node is a Perl "undef".

Sequence rule nodes¶

Some rules are sequence rules. Sequence rule nodes are also rule nodes. Everything said above about rule nodes applies to sequence rule nodes. Specifically, the arguments to the value actions for sequence rules are the per-parse-tree variable followed by the values of the child nodes in lexical order.

The difference (and it is a big one) is that in an ordinary rule, the right hand side is fixed in length, and that length is known when you are writing the code for the value action. In a sequence rule, the number of right hand side symbols is not known until node evaluation time. The rule evaluation closure of a sequence rule must be capable of dealing with a variable number of arguments.

Sequence semantics work best when every child node in the sequence has the same semantics. When that is not the case, writing the sequence using ordinary non-sequence rules should be considered as an alternative.

By default, if a sequence rule has separators, the separators are thrown away before the value action is called. (Separators are described in the section introducing sequence rules.) This means that separators do not appear in the @_ array of the rule evaluation closure which is the value action. If the value of the "keep" rule property is a Perl true value, separators are kept, and do appear in the value action's @_ array.

Null nodes¶

A null node is a special case of a rule node, one where the rule derives the zero-length, or empty string. When the rule node is a null node, the rule evaluation closure will be called with no child value arguments.

When a node is nulled, it must be as a result of a nullable rule, and the action name and action are those associated with that rule. An ambiguity can arise if there is more than one nullable rule with the same LHS, but a different action name. In that case the action name for the null nodes is that of the empty rule.

The remaining case is that of a set of nullable rules with the same LHS, where two or more of the rules have different action names, but none of the rules in the set is an empty rule. When this happens, Marpa throws an exception. To fix the issue, the user can add an empty rule. For more details, see the document on null semantics.

Reserved action names¶

Action names that begin with a double colon (""::"") are reserved. At present only the "::undef" reserved action is documented for use outside of the DSL-based interfaces.

::undef

A constant whose value is a Perl "undef". Perl is unable to distinguish reliably between a non-existent value and scalars with an "undef" value. This makes it impossible to reliably distinguish resolutions to a Perl "undef" from resolution problems. The ""::undef"" reserved action name should be preferred for indicating a constant whose value is a Perl "undef".

Explicit resolution¶

The recognizer's "closures" named argument allows the user to directly control the mapping from action names to actions. The value of the "closures" named argument is a reference to a hash whose keys are action names and whose hash values are references. Typically (but not always) these will be "CODE" refs.

If an action name is the key of an entry in the "closures" hash, it resolves to the closure referenced by the value part of that hash entry. Resolution via the "closures" named argument is called explicit resolution.

When explicit resolution is the only kind of resolution that is wanted, it is best to pick a name that is very unlikely to be the name of a Perl object. Many of Marpa::HTML's action names are intended for explicit resolution only. In Marpa::HTML those action names begin with an exclamation mark ("!"), and that convention is recommended.

Fully qualified action names¶

If explicit resolution fails, Marpa transforms the action name into a fully qualified Perl name. An action name that contains a double colon (""::"") or a single quote (""'"") is considered to be a fully qualified name. Any other action name is considered to be a bare action name.

If the action name to be resolved is already a fully qualified name, it is not further transformed. It will be resolved in the form it was received, or not at all.

For bare action names, Marpa tries to qualify them by adding a package name. If the "actions" grammar named argument is defined, Marpa uses it as the package name. Otherwise, if the "action_object" grammar named argument is defined, Marpa uses it as the package name. Once Marpa has fully qualified the action name, Marpa looks for a Perl object with that name.

Marpa will not attempt to resolve an action name that it cannot fully qualify. This implies that, for an action name to resolve successfully, one of these five things must be the case:

• The action name is one of the reserved action names.
• The action name resolves explicitly.
• The action name is fully qualified to begin with.
• The "actions" named argument is defined.
• The "action_object" named argument is defined.

Marpa's philosophy is to require that the programmer be specific about action names. This can be an inconvenience, but Marpa prefers this to silently executing unintended code.

If the user wants to leave the rule evaluation closures in the "main" namespace, she can specify "main" as the value of the "actions" named argument. But it can be good practice to keep the rule evaluation closures in their own namespace, particularly if the application is not small.

Types of Perl actions¶

Actions resolve in three ways: to reserved actions, to Perl rule evaluation closures and to Perl variable actions. The following are tried, in order.

• If an action name begins with a double colon (""::""), it will resolve to a reserved action, or not at all.
• If the fully qualified form of an action name is the name of a Perl subroutine in the symbol table, the action name resolves to the Perl subroutine. That subroutine then becomes a Perl rule evaluation closure.
• If the fully qualified form of an action name is the name of a Perl scalar variable in the symbol table, the action name resolves to the Perl variable. Note that, for this purpose, a Perl reference variable is considered as one type of Perl scalar.
• Other kinds of Perl objects in the symbol table that match the fully qualified name, such as arrays and hashes, are ignored. Note that, while resolution to arrays and hashes is not allowed, resolution to references to arrays and hashes is permitted.

Resolution to a Perl rule evaluation closure or to a Perl variable may came from explicit resolution. Explicit resolution always takes place via a reference, and requires an extra level of indirection. For resolution to a rule evaluation closure, the closure must be provided in the form of a reference to the closure. For resolution to a Perl variable, the variable has to be provided in the form of a reference to the variable. If the Perl variable is a reference, that means adding another level of indirection.

Modifying Perl variable actions¶

When resolution is to a Perl variable, it is possible to modify the value of the variable. In practice, this will usually be a bad idea. The Perl variable reference actions should be treated as read-only constants, and never modified.

This is because multiple resolutions to a Perl variable will always point to the same contents. Any modification to those contents will be seen by other users of that Perl variable. In other words, the modification will have global effect. For this reason modifying the referents of reference actions is almost always bad practice at the least, and is often an error.

For example, assume that actions are in a package named "My_Nodes", which contains a hash reference named "empty_hash",

        package My_Nodes;
        our $empty_hash = {};

It can be tempting, in building objects which are hashes, to start with a leaf node whose action is "empty_hash" and to add contents to it as the object is passed up the evaluation tree. But $empty_hash points to a single hash object. This single hash object will shared by all nodes, with all nodes seeing each other's changes. Worse, all Marpa parsers which use the same "My_Nodes" namespace will share the same hash object. An application which needs an action which produces an empty hash should have the action resolve to a Perl rule evaluation closure that returns "{}".

Visibility and resolution¶

When Perl closures and variables are used for the semantics, they must be visible in the scope where the semantics are resolved. The action names are usually specified with the grammar, but action resolution takes place in the recognizer's "value" method. This can sometimes be a source of confusion. For example, if a Perl closure is visible when the action is specified, but goes out of scope before the action name is resolved, resolution will fail.

Action object constructor¶

If the grammar's "action_object" named argument has a defined value, that value is treated as the name of a class. The action object constructor is the "new" method in the "action_object" class.

The action object constructor is called in the Tree Setup Phase. The return value of the action object constructor becomes the per-parse-tree variable. It is a fatal error if the grammar's "action_object" named argument is defined, but does not name a class with a "new" method.

Resolution of the action object constructor is resolution of the action object constructor name. The action object constructor name is formed by concatenating the literal string ""::new"" to the value of the "action_object" named argument.

All standard rules apply when resolving the action object constructor name. In particular, bypass via explicit resolution applies to the action object constructor. If the action object constructor name is a hash key in the evaluator's "closures" named argument, then the value referred to by that hash entry becomes the action object constructor.

If a grammar has both the "actions" and the "action_object" named arguments defined, all action names except for the action object constructor will be resolved in the "actions" package or not at all. The action object constructor is always in the "action_object" class, if it is anywhere.