intro to the lua c api, part 1

the state, the stack, and running lua from your own c++

what youll have by the end

a tiny lua repl you wrote yourself, type a line of lua, see what came back, and get a real error with a trail when it goes wrong

> 1 + 2
3
> x * 2, "hi", nil
20  "hi"  nil

no library, no module, just your c++ and the lua api

where this sits

this is part 1 of 3, and the whole series is the api on its own, the functions, what they do, and small programs that use them

  • part 1 the state, the stack, and running lua from c++
  • part 2 c++ functions lua can call, and tables
  • part 3 userdata, the registry, and coroutines

each part ends on one small project, 90 minutes at most

the big picture

lua is not a program you talk to from the outside, its a library, a pile of c code you link into your own program

the c api is the set of functions that lets your c++ and lua talk, both ways, your code drives lua, and lua calls your code

this part is the first half, your c++ driving lua, making one, giving it values, running code in it, and reading back what happened

like a serving hatch

two kitchens, yours in c++ and lua next door, with one hatch in the wall between them

you cant reach into their kitchen and grab a pot, and they cant reach into yours, everything goes through the hatch, one plate at a time, stacked up

that stack of plates is the lua stack, and almost every call in this deck is either putting a plate on it or taking one off

the map

diagram

your program only ever touches the stack, everything else inside L it reaches through the stack

section 1, a first program

what you link

lua ships as plain c, so you need two things, the headers and the library

#include <lua.hpp>

lua.hpp pulls in the three headers you use, lua.h the core api, lauxlib.h the helpers, lualib.h the std libs, all wrapped in extern "C"

extern "C" tells the c++ compiler these are c functions, so it looks for their plain c names when it links, without it you get a wall of undefined symbol errors

building it, xcode

build liblua once, open lua-5.5.1/lua.xcodeproj, pick the liblua scheme, build it in release, it lands at lua-5.5.1/build/Release/liblua.a

then in your own command line tool project

  • build settings, header search paths add lua-5.5.1/source
  • build phases, link binary with libraries +, add other, pick liblua.a

or straight from a terminal at the repo root

clang++ -std=c++20 -I lua-5.5.1/source main.cpp lua-5.5.1/build/Release/liblua.a -o host

building it, vs2022

this slide gets written from the windows pc, it comes next time finn is on it

the smallest host

#include <lua.hpp>

int main()
{
	lua_State *L = luaL_newstate();
	luaL_openlibs(L);

	luaL_dostring(L, "print('hello from lua', 1 + 2)");

	lua_close(L);
	return 0;
}
hello from lua	3

five lines of lua api and you have a working scripting language inside your program

the three calls that frame every host

luaL_newstate() makes a brand new lua and hands back a lua_State *, the whole lua world in one pointer, its globals, its libs, its stack. its opaque, you never look inside, you just pass it to every other call

luaL_openlibs(L) loads the std libs, print, string, math, table and the rest, a new state starts with nothing, not even print

lua_close(L) tears the whole lua down and frees every bit of memory it used

by habit the pointer is always called L, and finn-style lets it stay L too, a rare single letter name bc the whole lua world agrees on it

section 2, the stack

the stack, the one idea

a stack of values, the plates on the hatch

you push a value to put it on top, you pop to take values off the top, and you can read any slot by its number without taking it off

lua and your c++ both use it, so its how every value crosses over, a number, a string, a function, a whole table

pushing values on

one function per kind of value, each one grows the stack by exactly one

lua_pushnil(L);
lua_pushboolean(L, 1);
lua_pushinteger(L, 42);
lua_pushnumber(L, 3.5);
lua_pushstring(L, "hello");

lua_Integer is a 64 bit int and lua_Number is a double, the only two number kinds lua has

a string gets copied when you push it, lua never keeps your pointer, so your buffer can die straight after

indices, two ways to count

say we pushed "hello", then 400, then 300

up from the bottom, 1 is "hello", 2 is 400, 3 is 300, handy for things that sit still, their number never changes while you push on top

down from the top, -1 is 300, -2 is 400, -3 is "hello", and -1 always means whatever is on top right now, so its the one for the thing you just pushed

the three stack chores

lua_gettop(L) how many values are on the stack, which is also the index of the top one

lua_settop(L, n) make the stack exactly n tall, chopping values off or padding with nil, lua_settop(L, 0) empties it

lua_pop(L, n) throw away the top n values

lua_checkstack(L, n) asks for room for n more, lua promises at least 20 free slots (LUA_MINSTACK), so only a loop pushing hundreds needs it

whats in a slot

int type = lua_type(L, -1);
printf("%s\n", lua_typename(L, type));

lua_type hands back a number, LUA_TNIL LUA_TBOOLEAN LUA_TNUMBER LUA_TSTRING LUA_TTABLE LUA_TFUNCTION and a few more

lua_typename turns that number into a word, "number", "string"

luaL_typename(L, idx) does both in one go, the luaL_ prefix just means its a helper from lauxlib.h, built out of the core calls

the lua_is family

lua_isnumber lua_isstring lua_isinteger and friends

careful, they ask could this be read as one, not is it one

lua_pushstring(L, "10");
lua_pushinteger(L, 10);
lua_pushstring(L, "ten");
isnumber  1 1 0
isstring  1 1 1
type      string number string

"10" passes isnumber and 10 passes isstring, bc lua happily converts between the two. when you need the real type, ask lua_type

reading values back

int ok = 0;
lua_Integer n = lua_tointegerx(L, 3, &ok);

lua_tointegerx lua_tonumberx reads a slot as a number, the x version fills ok with 1 if it worked and 0 if it didnt

tointegerx on "ten"  n = 0, ok = 0
tointegerx on "10"   n = 10, ok = 1

the plain lua_tointeger(L, idx) is the same with no ok, it just gives 0 for anything that isnt a number, so a real 0 and a failure look the same

lua_toboolean follows lua truth, only nil and false are false, so a 0 reads as true, which catches out everyone coming from c

lua_tolstring

size_t len = 0;
const char *str = lua_tolstring(L, -1, &len);

reads a slot as a string and fills len with its length, lua_tostring(L, idx) is the same with no length, and both give NULL for a table, a nil or a function

the two tolstring traps

the pointer lives on the stack str points into lua memory, it stays good only while that string is still on the stack, pop it and the pointer can dangle. need it longer, copy it into a std::string

it changes a number in place read a number as a string and the slot becomes a string

before  number
got "10" len 2
after   string

harmless most of the time, but part 2 has a loop where it breaks things, so remember it now

moving things around

lua_pushvalue(L, idx) pushes a copy of the slot at idx onto the top

lua_remove(L, idx) takes the slot at idx out and slides everything above it down one

lua_pushvalue(L, 1);
lua_remove(L, 1);

there are a few more, lua_insert lua_replace lua_rotate lua_copy, same idea, look them up when you need one

still with me ?

so far

  • L is the whole of lua, you pass it to everything
  • the stack is the hatch, push to put on, pop to take off, read any slot by index
  • 1 up from the bottom, -1 down from the top, same slots, two names
  • lua_type says what a slot is, lua_to* reads it

next, a tool to look at the stack whenever we like

section 3, the stack dumper

what we want

a function we can drop anywhere, dump_stack(L), that prints every slot, both indices, the type and the value

stack, 5 tall
   5   -1  string    "hello"
   4   -2  number    3.5
   3   -3  number    42
   2   -4  boolean   true
   1   -5  nil       nil

every bug in this series is a stack bug at heart, so this is the tool we reach for from here on

one slot

//
//	prints the value at the given index, no newline
//
void print_slot(lua_State *L, int _idx)
{
	switch (lua_type(L, _idx))
	{
		case LUA_TNIL:
			printf("nil");
			break;
		case LUA_TBOOLEAN:
			printf(lua_toboolean(L, _idx) ? "true" : "false");
			break;
		case LUA_TNUMBER:
			if (lua_isinteger(L, _idx))
				printf("%lld", (long long)lua_tointeger(L, _idx));
			else
				printf("%g", lua_tonumber(L, _idx));
			break;
		case LUA_TSTRING:
			printf("\"%s\"", lua_tostring(L, _idx));
			break;
		default:
			printf("%s", luaL_typename(L, _idx));
			break;
	}
}

switch on the type, read it the right way, anything else just prints its type name

lua_tostring only runs on something that is already a string, so a number never gets turned into one behind our back, and dumping the stack never changes the stack

the loop

//
//	prints every slot on the stack, top first
//
void dump_stack(lua_State *L)
{
	int top = lua_gettop(L);

	printf("stack, %d tall\n", top);
	for (int i = top; i >= 1; i--)
	{
		printf("  %2d  %3d  %-8s  ", i, i - top - 1, luaL_typename(L, i));
		print_slot(L, i);
		printf("\n");
	}
}

top down so it reads like the plates, i - top - 1 turns the up index into the down one

both functions sit in an anonymous namespace, a namespace with no name, so nothing outside this file can see them, finn-style uses it over static for file only helpers, and the _ on _idx marks an argument

exercise 1, the dumper

write print_slot and dump_stack, then a main that pushes a mix and dumps after each move

int main()
{
	lua_State *L = luaL_newstate();

	// todo, push a nil, a bool, an int, a double, a string
	// todo, dump, pushvalue 3, dump, remove 1, dump, settop 0, dump

	lua_close(L);
	return 0;
}

pseudo

  1. print_slot switches on lua_type and prints each kind its own way
  2. dump_stack loops from the top down, printing both indices, the type name and the slot
  3. main pushes five values and dumps after every change

tip the pushvalue 3 dump should show a second 42 on top, if it doesnt, check you passed a positive index, 3 counts from the bottom

section 4, running lua

luaL_dostring is two calls

luaL_dostring(L, "print('hi')");

is just shorthand for

luaL_loadstring(L, "print('hi')") || lua_pcall(L, 0, LUA_MULTRET, 0);

load the code, then call it, and splitting them is where all the control comes from

loading compiles the code and pushes it as a function, a chunk, nothing has run yet

load status 0, top is a function, stack 1 tall

luaL_loadfile(L, "game.lua") is the same from a file, luaL_loadbuffer from a buffer with a length and a name

lua_pcall

lua_pcall(L, nargs, nresults, msgh);

nargs how many arguments are sitting above the function, 0 for a chunk

nresults how many results you want back

msgh where the error handler sits, 0 for none, section 6 has this one

it pops the function and its arguments, runs it, and pushes the results where the function was

how many results

luaL_loadstring(L, "return 1 + 2, 'three'");
lua_pcall(L, 0, LUA_MULTRET, 0);
pcall status 0, stack 2 tall, 3 and three

LUA_MULTRET means keep however many came back, then lua_gettop tells you how many that was

ask for a number instead and lua makes it fit, extra results get dropped, missing ones get padded with nil

how it flows

diagram

the status codes

both luaL_load* and lua_pcall hand back a status

code when
LUA_OK it worked, its 0
LUA_ERRSYNTAX the code didnt compile, from a load
LUA_ERRFILE luaL_loadfile couldnt open the file
LUA_ERRRUN it compiled but failed while running
LUA_ERRMEM lua ran out of memory
LUA_ERRERR the error handler itself failed

you mostly just check for LUA_OK, the rest are for when you want to say why

the error is a value

when it isnt LUA_OK, the error message is sitting on top of the stack, in place of the chunk or the results

if (lua_pcall(L, 0, 0, 0) != LUA_OK)
{
	printf("%s\n", lua_tostring(L, -1));
	lua_pop(L, 1);
}
syntax  status 3  [string "x = = 1"]:1: unexpected symbol near '='
runtime status 2  [string "local t = nil..."]:2: attempt to index a nil value (local 't')

read it, then pop it, or it sits there forever

naming a chunk

that [string "local t = nil..."] is lua naming the chunk after its own code, ugly in a long one

luaL_loadbuffer takes a name, start it with = for a name as is, or @ for a file name

luaL_loadbuffer(L, code, strlen(code), "=repl");
repl:2: attempt to index a nil value (local 't')

luaL_loadfile names the chunk after the file for you, game.lua:2:

exercise 2, run a file

a host that runs the file named on the command line, ./host game.lua, and reports any error

int main(int _argc, char **_argv)
{
	// todo, no file given, print how to use it and stop
	// todo, make a state, open the libs
	// todo, load the file, run it, print the error if either failed
	// todo, close the state
}

pseudo

  1. _argc under 2 means no file, print usage: host <file>
  2. luaL_loadfile with _argv[1], then lua_pcall only if that was LUA_OK
  3. not LUA_OK at either step, print the top of the stack

tip try a missing file, a file with a typo, and one that errors while running, you should see LUA_ERRFILE, LUA_ERRSYNTAX and LUA_ERRRUN messages, all off the top of the stack

globals, in and out

lua_pushinteger(L, 3);
lua_setglobal(L, "lives");

lua_setglobal pops the top value and stores it as a global, so a script can now print(lives) and get 3, this is how a host hands a script its settings before running it

int type = lua_getglobal(L, "score");
lua_Integer score = lua_tointeger(L, -1);
lua_pop(L, 1);

lua_getglobal pushes the global and hands back its type, a missing one pushes nil and says LUA_TNIL. a get always pushes, so a get always needs a pop

calling a lua function

say a script defined this

function damage(hp, hit)
	local left = hp - hit
	return left, left <= 0
end

calling it from c++ is the same push then pcall as a chunk, with arguments this time

lua_getglobal(L, "damage");
lua_pushinteger(L, 30);
lua_pushinteger(L, 45);
lua_pcall(L, 2, 2, 0);

reading several results

the results land in order, the first result deepest, the last on top

lua_Integer left = lua_tointeger(L, -2);
int dead = lua_toboolean(L, -1);
lua_pop(L, 2);
left -15, dead yes, stack 0 tall

asked for 2, so read -2 and -1, then pop both and the stack is back how we found it

exercise 3, call a lua function

write rules.lua with damage in it, run it, then call damage(30, 45) from c++ and print both results

pseudo

  1. luaL_dofile the rules, so damage exists as a global
  2. get the global, push the two arguments
  3. lua_pcall with 2 in and 2 out, print the error if it failed
  4. read -2 and -1, pop 2

tip dump the stack right before the pcall, you should see the function at the bottom with 30 and 45 on top of it, if the function isnt there, rules.lua didnt run

section 5, when it goes wrong

an error with no trail

function hurt(enemy)
	enemy.hp = enemy.hp - 10
end

function tick()
	hurt(nil)
end

tick()
game.lua:2: attempt to index a nil value (local 'enemy')

we know where it broke, but not who called it, and in a real game hurt gets called from a hundred places

the message handler

the last argument to lua_pcall is the message handler, a function lua calls with the error before it gives up on the call

thats the key bit, the call is still alive when the handler runs, so the handler can still see who called who

once lua_pcall returns, all of that is gone, so the trail has to be built inside the handler

//
//	adds a traceback to the error on top of the stack
//
int traceback(lua_State *L)
{
	luaL_traceback(L, L, lua_tostring(L, 1), 1);
	return 1;
}

a c++ function lua can call, part 2 is all about that shape. the error comes in at 1, luaL_traceback pushes it back with a trail added (the last 1 skips our handler itself), and return 1 says we left one result

using it

lua_pushcfunction(L, traceback);
int status = luaL_loadfile(L, "game.lua");
if (status == LUA_OK)
	status = lua_pcall(L, 0, 0, 1);
if (status != LUA_OK)
	printf("%s\n", lua_tostring(L, -1));
lua_settop(L, 0);

push the handler first so it sits at 1, under the chunk, then tell lua_pcall it is at 1

game.lua:2: attempt to index a nil value (local 'enemy')
stack traceback:
	game.lua:2: in global 'hurt'
	game.lua:6: in global 'tick'
	game.lua:9: in main chunk

lua_call, and why a host always pcalls

lua_call(L, nargs, nresults) is lua_pcall with no catching, an error has nowhere to go

PANIC: unprotected error in call to Lua API ([string "local t = nil..."]:2: attempt to index a nil value (local 't'))

with nothing above it to catch the error, lua calls its panic function, prints that, and aborts the whole program

so the rule, the host only ever runs lua through lua_pcall, lua_call is for code that is already inside a pcall, which part 2 gets to

section 6, the project, a mini repl

what it does

about 60 to 75 minutes, a repl, read a line, run it, print what came back, loop

> 1 + 2
3
> x = 10
> x * 2, "hi", nil
20  "hi"  nil
> function hurt(t) return t.hp - 1 end
> hurt(nil) + 1
repl:1: attempt to index a nil value (local 't')
stack traceback:
	repl:1: in global 'hurt'
	repl:1: in main chunk
> exit

every piece of it is from this deck

the return first trick

1 + 2 isnt valid lua on its own, its an expression, a value, not a statement, something to do

so try every line as return <line> first, if that loads, its an expression and the results come back

if it doesnt load, pop that error and load the line as it is, x = 10 is a statement and returns nothing

one line, start to finish

diagram

the project

int main()
{
	lua_State *L = luaL_newstate();
	luaL_openlibs(L);

	char line[1024];
	while (true)
	{
		// todo, print "> ", read a line, stop on end of input or exit
		// todo, empty the stack, push the handler
		// todo, load it, return first, run it, print the results or the error
	}

	lua_close(L);
	return 0;
}

pseudo

  1. fgets the line, cut the newline off, exit breaks the loop
  2. lua_settop(L, 0) then push traceback, so the handler is always at 1
  3. a load_line helper, return plus the line through luaL_loadbuffer named =repl, on a fail pop and load the line bare
  4. loaded, lua_pcall(L, 0, LUA_MULTRET, 1)
  5. failed anywhere, print the top, worked, print_slot every slot from 2 to the top

tip std::string("return ") + _line builds the expression, and luaL_loadbuffer wants its length, .size(). everything above slot 1 after the pcall is a result, so the loop starts at 2

section 7, wrapping up

the new calls

call what it does
luaL_newstate luaL_openlibs lua_close make, fill and end a lua
lua_push* lua_pushvalue put a value or a copy on top
lua_gettop lua_settop lua_pop lua_remove read or change the stack
lua_type lua_to* what a slot is, and read it
luaL_load* compile code, push it as a function
lua_pcall run it, catch any error
lua_setglobal lua_getglobal pop into a global, push a global
luaL_traceback an error plus who called who

why its built this way

  • load and call split, so a syntax error and a runtime error come out of different steps and you can tell them apart
  • always lua_pcall, an error with nothing to catch it takes the whole program down
  • the handler at a fixed slot, pushed first so its index never moves, 1 every time
  • the dumper reads by type, so it never turns a number into a string behind your back
  • finn-style, the anonymous namespace over static, _ on arguments, the three line header above every function, and L stays L

recap

  • lua is a library, luaL_newstate makes one, luaL_openlibs fills it, lua_close ends it
  • every value crosses over on the stack, 1 up from the bottom, -1 down from the top
  • a get or a load pushes, so something always has to pop
  • loading compiles code into a function, lua_pcall runs it and leaves results or an error on top
  • a message handler runs before the call unwinds, so its the only place a traceback can be built

have a go

make the repl take code over several lines

> for i = 1, 3 do
>> print(i)
>> end
1
2
3

pseudo

  1. when a load fails, look at the message, an unfinished chunk ends in <eof>
  2. keep the text, print >> , read another line, add it on with a newline, try again
  3. any other load error is a real one, print it like before

tip the real message is repl:1: 'end' expected near <eof>, so check the end of the string, not the start

whats next

part 2, c++ functions and tables

so far lua only calls lua, next lua calls your c++, a function with the same shape as our traceback, arguments checked properly, errors that read well

then tables, how anything bigger than a few values crosses the hatch, and where that lua_tolstring trap finally bites