blog/ Lexical Environments in Throff

I finally have lexical environments sorted out in Throff.

Never again.

Lexical scoping has been by far the hardest part of implementing Throff. Each time I think I have it working, I hit an edge case and realise I need to change the design again. But this time around, I think I have finally got a working design.

Previously, every data item in Throff held a pointer to its lexical environment. This was needed, because the current lexical scope was always taken from the current instruction (or previous, in the case of macros). This was needed to choose the correct environment when Throff created a new data item. e.g.

ADD 1 1

2

when Throff created the result "2", it used the environment from the previous "1". Having an environment isn't so useful for numbers, but it was important when evaluating strings. e.g.

EVAL STRING-CONCATENATE "PRINTLN" "GREETING"
SETLEX GREETING => ->STRING [ PRINTLN HELLO_WORLD ]

EVAL does need a lexical environment to lookup the variable GREETING so it can print out HELLO_WORLD. Previously, it would use the environment of the string. But this won't work if the string was created in a different namespace. The lookup for "GREETING" will happen in a different namespace to the current one. There are similar problems dealing with continuations and error handling

Now, EVAL it requires an environment as an argument, which has the benefit of allowing you to evaluate code in a different environment to the one that the code appears in.

EVAL text IN ENVIRONMENTOF token

Since TOKEN and CODE (and LAMBDA) are now the only datatypes allowed to have an environment, you will have to explicitly get the environment with

ENVIRONMENTOF abcd TOK

TOK "quotes" abcd, pushing it onto the data stack instead of treating it like a variable to be looked up and replaced with a value. ENVIRONMENTOF extracts the lexical environment from the TOKEN "abcd". Under the hood, an environment is just a HASH, and so you can use ordinary hash functions on it. So for instance, you can dump the current lexical environment with

ITERATE [ PRINTLN ] KEYS ENVIRONMENTOF abcd TOK

This is also a handy way to deal with namespaces, although the semantics of updating an immutable namespace get a bit weird.