::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:
class Program.Parse.Public public Gilda
type Tokenizer is &A simple way to tokenize strings.
Blank string, &Blank characters to be ignored.
Comment string, &Characters beginning a comment (Space separated).
Quote string, &Language format for quoted constants.
Unary string, &List of unary operators.
Number string, &Language format for numeric constants.
Name string, &First and remaining characters in a name.
Operator string, &List of operators (Space separated).
Keyword string, &List of reserved keywords (Space separated).
Delimiter string, &List of delimiters.
Special string, &List of special characters.
Begin string, &Begin a grouping.
End string :End a grouping.
type Emit.Type is &
Emit string, &
Type Token.Type
type Token.Type in Comment, Quote, Unary, Number, Name, Operator,
Keyword, Delimiter, Special, Begin, End
:
:...............................................................................
::::::::::::::::::::::::: program/parse/Tokenizer ::::::::::::::::::::::::::::
:
sequence TOKENIZER..Tokenizer: Tokenize a line of text.
entry Tokenizer, &Parsing configuration
Line string :Text to tokenize
exit Token Emit.Type :Each token
:
:...............................................................................
:
function GILDA..Tokenizer: Establish tokenization settings for Gilda.
exit Tokenizer :Parsing configuration
:
:...............................................................................
:
method SCAN..Tokenizer pure: Get the next token in a string.
entry Tokenizer, &Parsing configuration
Text string :Text to be scanned
change Index word : In - First character to begin scanning
:Out - Character after the token.
exit Token string, &Emitted text or empty if end of line
Type Token.Type :Token type; 0 if unknown or end of line
:
:...............................................................................
:
method SCAN.QUOTE..Tokenizer pure: Parse a quoted string.
entry Tokenizer, &Parsing configuration
Text string :Text to be scanned
change Index word : In - First character to begin scanning
:Out - Character after the token
exit Token string :Emitted text or empty if not a quote
:
: A quote token is a Gilda style quote with '"' or '"' delimiters.
:...............................................................................
:
method CHOMP..Tokenizer pure: Parse a token and remove it from the front of a string.
entry Tokenizer :Parsing configuration
change Text string :Text to eat
exit Token string, &Next token
Type Token.Type :Kind of token returned
:
:...............................................................................
:
function FORM..Emit.Type: Display a token type and emitted text.
entry Emit.Type, &Type of token
Format = "" string :String format for $Emit
exit Form string :Formatted token; <$Type>: <$Emit>
:
:...............................................................................
:::::::::::::::::::::: program/utility/Program.Utility :::::::::::::::::::::::
:
function FORM.DATE: Format the current time and date.
exit Date string :Formed like: Sat April 16 15:27:03
:
:...............................................................................
:
function HEAP.TO.LIST: Parse a heap into a delimited text list.
entry Heap..string, &A heap of source strings
Delimiter = "," string, &Delimiter to separate the strings
Index = 0 word :Index past the first string.
exit List string :Strings separated by the delimiter
:
:...............................................................................
:
function LIST.TO.HEAP: Parse a text list into a heap.
entry List string, &Source list of items
Delimiter = ',' byte :Delimiter between items
exit Heap Heap..string :Each item in the list (no outer blanks)
:
: Leading and trailing spaces and tabs are removed from each item.
:...............................................................................
:::::::::::::::::::::::::::: program/Data.File :::::::::::::::::::::::::::::::
:
sequence READ.DATA.STRING: Read each item in a String data file.
entry Source string :Relative path of the data file
exit Data string :Text string
:
:...............................................................................
:
sequence READ.DATA.NUMBER: Read numeric values from a data file.
entry Source string :Relative path of the data file
exit Data string :Numeric value as text
:
:...............................................................................
:
method READ.DATA.SIZE: Read and parse a data file to determine the size.
entry Source string, &Source data file
Base word :Type of data
exit Size word :Number of entries in the file
:
:...............................................................................
:
method CHECK.DATA.CRC: Validate the CRC for a data set.
entry Comment string, &' '+ Integer
Expected word :Expected value
:
:...............................................................................
end