If we had a Bitlash Best Hack of the Month award, this month's would surely go to Brett Hagman for his superb software prototype which demonstrates running Bitlash scripts off the uMMC shield. Check out Brett's post at Rogue Robotics, then please return here for some technical comments.

The prototype Bitlash_uMMC.pde is a clever (and very well-documented!) use of the new user-functions facility in Bitlash 1.1. The user function “exec()” opens a file from the storage card and executes the text there as Bitlash commands by passing one line at a time to the Bitlash doCommand() function.

What a clever combination of user functions + doCommand()! Unfortunately, while the prototype works (and bravo to Brett), danger lurks just beneath the surface, due to the fact that doCommand() is not reentrant.

Let's dig into the problem a bit before walking through a proposed solution. I hope I may safely quote the first line of the Wikipedia entry on reentrancy, which summarizes the issue well:

A computer program or routine is described as reentrant if it can be safely called again before its previous invocation has been completed…

Here is where the problem arises: when your user function is called, you are deep in the bowels of the interpreter stack somewhere under doCommand() . So calling doCommand() from a user function will end up calling doCommand() while it's already active on the stack.

Now, doCommand() was never designed to be reentrant since, among other things, it uses setjmp()/longjmp() to set the global error handling context. Have a look at the error handling in src/bitlash-error.c; you will see that all the errors end with a longjmp() back to the originating doCommand().

I suspect there would be no shortage of interesting debugging experiences if, for example, a macro running from eeprom runs an external script in the way the prototype does, and then throws an error after the script is done running (while the now-stale setjmp() context is deep in the stack). Wierdly nondeterministic behavior in the middle of an error case is never fun to track down.

This is why the documentation for Bitlash user functions recommends against calling doCommand() or runBitlash() from a user function.

So this approach is likely to be pretty fragile under error cases at least, without work to make doCommand() reentrant. Is there a cheaper approach?

The obvious and ideal approach would be to integrate the external storage as an input stream to the parser, the same way ram and eeprom are used today.

Implementing scripts on external storage as first-class citizens in the core would enable external scripts to call macros in eeprom, and vice versa. The net effect would be very powerful, but at the price of modifying the parser to enable parseid() to look up identifiers in the external store, and fetchc() to get characters from it like we do from the eeprom.

While this might be the “right” approach in the long run, there is somewhat risky work involved, and I have a proposal for a simpler approach which might meet Brett's requirements.

The basic idea is this: there is a place in the interpreter where Bitlash silently ignores references to undefined macros. (It is not a fatal error not to know a word.) Instead of ignoring the undefined reference, why not look it up and execute it from the external storage instead?

There is a second part, too. Since calling doCommand() from within Bitlash is a problem, let's organize the solution so the calls to Bitlash to execute the external script's commands from safely from outside Bitlash in the call tree.

The second part is easy using an approach one might call co-tasking: add a new procedure called runScriptTask() to handle the dirty work of calling doCommand() from outside Bitlash, in the loop() function, at the same level as runBitlash(). In honor of Brett's work we might call it Bhagman's Daemon:

loop(void) {
	runBitlash();		// Bitlash gets time
	runScriptTask();	// and so does our script running co-task
}

Think of runScriptTask() as a daemon waiting to process requests. This is where the fetch-execute loop from Brett's sdexec() function will go.

Now about those requests: we need a way to communicate script running requests to the runScriptTask(). Let's add an entry point named runExternalScript() to accept requests. The simplest implementation that comes to mind is a flag and buffer scheme something like this (warning uncompiled code):

#define IDLEN 12				// duplicating the Bitlash definition.  sigh.
byte scriptRequest;				// flag=1 when a script request is pending
char scriptName[IDLEN+1];		// IDLEN is 12 to allow "filename.txt" ;)

// script-run request hook
void runExternalScript(char *name) {
	if (scriptRequest) Serial.println("OOPS can't nest external script calls");
	else {
		strcpy(scriptName, name);	// capture the name from the bitlash buffer
		scriptRequest = 1;			// set flag for our top-level runScriptTask
	}
}

// script running task
void runScriptTask(void) {
	if (scriptRequest) {
		/* ... this is where the execute loop goes ... */
		/* ... open the file named in scriptName[] ... */
		/* ... call doCommand for each line        ... */
		scriptRequest = 0;		// clear the lock AFTER execution
	}
}

loop(void) {
	runBitlash();		// Bitlash gets time
	runScriptTask();	// and so does our script running co-task
}

This could of course be improved on but I think the general idea is clear.

What remains is the matter of arranging for a callout to runExternalScript(). Take a deep breath and open the file src/bitlash-interpreter.c; all the way at the bottom you will find the procedure doMacroCall():

void doMacroCall(int macroaddress) {
char op = sym;					// save sym for restore
	if (macroaddress >= 0) {
		/* ...code to handle eeprom macro call here... */
	}
}

You'll note there is no else clause; this is where undefined macro references are silently ignored (when macroaddress < 0).

Now I have to let you in on some insider information. At this point in the code, for one brief moment, the parsed but as-yet-unrecognized identifier is in a global buffer named “idbuf[]”, which is defined in src/bitlash.h:

// Temporary buffer for ids
#define IDLEN 12
extern char idbuf[IDLEN+1];

So the code change to pass the unknown identifier along for consideration by the scriptRunTask() is “else runExternalScript(idbuf);”:

void doMacroCall(int macroaddress) {
char op = sym;					// save sym for restore
	if (macroaddress >= 0) {
		/* ...code to handle eeprom macro call here... */
	}
	else runExternalScript(idbuf);	// unknown id: try to exec it from mmc
}

So, from the top, here is the proposed flow for the somewhat-safer method of running scripts from external storage:

• The user types an external script name on the Bitlash console
• Eventually runBitlash calls doCommand and the name is parsed
• It's not found in eeprom so we fall into the new “else” in doMacroCall
• doMacroCall passes the name to runExternalScript, which saves it
• Eventually runBitlash returns to loop
• loop calls runScriptTask
• runScriptTask finds it has work to do and executes the script like sdexec()

Brett, this has gotten quite lengthy, hasn't it, but I hope you will find some useful thoughts here. I'd be happy to respond to comments here, or we can cross post back and forth.

Cheers!

-br