The Scanner

What does the scanner recognize as a word? A word starts with any character and ends at the first special character. Thus

X@J+

would be broken into

X @J +

In practice, most words are separated by spaces - but this is not always convenient. Words have arbitrary length. The leading characters are stored in the dictionary and the rest discarded.

The convention the words may start with special characters increases the set of words with mnemonic value. For example

+ +1 +FIX +VECTOR

may represent different addition operators. And

IF .IF #IF

different forms of a conditional operator.

In the first case the primary mnemonic value is carried by the special character; in the second by the alphabetic word.

Period (.) is considered alphabetic so that it may serve as a break character and decimal point. Thus

3.14 COSMIC.RAY

are both words.

The scanner depends upon a character pointer to tell it where to read next. A word may change this pointer and thus alter the sequence of input words. This is the way definitions and backward branches are implemented. The user is free to establish other conventions if he wishes. Manipulated the character pointer permits a remarkably simple and independent scanner.

For example, input is initially expected from the keyboard. The first word would probably cause input to be read from disk - expanding the dictionary. Other words might come from definitions stored in core. The scanner itself is independent of the source of its input, and freely intermingles all 3 sources (Fig. 4).

The Scanner

The scanner accepts input indiscriminately from keyboard, disk and dictionary.