Test Suite for Commodore BASIC 2.0 Tokenization
===============================================

This test suite is written in the format of Falderal 0.10.

    -> Tests for functionality "Tokenize Commodore BASIC 2.0 program"

Fundamentals of Tokenization
----------------------------

Rudimentary program.

    | 10 print
    = 010807080a0099000000

Untokenizable lowercase ASCII becomes single-case (uppercase) PETSCII.

    | 10 swish
    = 01080b080a005357495348000000

Uppercase ASCII becomes PETSCII graphic symbols.

    | 10 PRINT
    = 01080b080a00d0d2c9ced4000000

All extra spaces after line number are stripped.

    | 10     print
    = 010807080a0099000000

Any spaces before line number are stripped.

    |    10 print
    = 010807080a0099000000

Tokens can successively follow one another.

    | 10 printa$
    = 010809080a00994124000000

Tokens can successively follow one another, even if syntax error.

    | 10 printtofor
    = 010809080a0099a481000000

The number of spaces occurring between tokens is preserved.

    | 10 print to   for  to
    = 010810080a009920a4202020812020a4000000

Strings in PETSCII.

    | 10 print "hello"
    = 01080f080a0099202248454c4c4f22000000

Multiple program lines.

    | 10 a=7
    | 20 a=a*2
    | 30 goto 20
    = 010809080a0041b237001308140041b241ac32001c081e0089203230000000

Certain control sequences in curly braces are recognized.

    | 1390 print"{clr}{home}{down}{rght}{up}{left}"
    = 01080f086e0599229313111d919d22000000

    | 1390 print"{red}{cyn}{blu}{grn}{pur}{yel}{wht}{blk}"
    = 010811086e0599221c9f1f1e9c9e059022000000

    | 1390 print"{SHIFT-@}{CBM-+}"
    = 01080b086e059922baa622000000

`rvs on` and `rvs off` aren't supported by `petcat` anymore, it seems.
Instead they are `rvon` and `rvof`.

    | 1390 print"{rvon}{rvof}"
    = 01080b086e059922129222000000

Unterminated string literals are allowed, and computed correctly.

    | 1400 print"hi
    | 1410 print
    = 01080a08780599224849001008820599000000

Inside a double-quoted string, nothing gets tokenized.

    | 1400 print"port
    | 1410 print
    = 01080c0878059922504f5254001208820599000000

Inside a REM, nothing gets tokenized.

    | 1 rem and
    = 01080b0801008f20414e44000000