binding c++ to lua, part 4

lua values held from c++, a function c++ keeps and calls later, a table it reads, and an event bus built on both

how this one works

questions are mixed in with the slides, and the deck waits at each one until you answer it

  • you get 5 tries, after a miss you get a hint and a show answer button
  • a first try is worth 1, each retry takes 0.2 off, and showing the answer is worth 0
  • every answer comes with the why, read it even when you got it right
  • the next button under each slide, the arrows along the top, a swipe or the arrow keys all move between slides, and your progress stays on this device

what youll have by the end

the part 3 classes, plus the other direction, lua things that c++ holds on to

  • lua_ref_t, a ticket for any lua value, so the gc keeps it while c++ needs it
  • refs that go dead when the state closes, instead of crashing your shutdown
  • lua_function_t<int(int)>, a lua function you call like a c++ one, and an error never throws out of it
  • a lua function wherever c++ wants a std::function, so game code never sees lua.h
  • an event bus, events.on("hit", fn), where one broken listener cannot stop the rest
  • lua_table_t, player["hp"] = 15 from c++, and std::vector and std::map both ways
  • a settings.lua read into a c++ struct, with errors that name the key

where this sits

part 1 built the host, part 2 generated the wrappers, part 3 decided who owns each object

every arrow so far went one way, c++ handed something to lua, and lua used it during a call

this part turns round, a script hands c++ a function, and c++ calls it later, a frame later, a minute later, long after the call that handed it over has returned

every snippet, output and error here came out of a real build against the lua 5.5.1 in this repo, built with msvc this time, so the compiler errors read differently from parts 2 and 3

the big picture

events.on("hit", function(target, damage)
	print(target:describe() .. " took " .. damage)
end)

the script hands over a function, then events.on returns, and the script carries on

some time later the goblin gets hit, and c++ has to call that function

so c++ has to keep a lua function somewhere, between calls, and that is the whole problem of this part, c++ has nowhere to put a lua value

like a coat check

you cannot take your coat into the theatre, so you hand it over at the desk and get a numbered ticket

the ticket is not the coat, it is a small number you can pocket, and the desk keeps the coat safe for as long as the ticket is out

hand the ticket back and the coat goes, lose the ticket and the coat sits there forever

here the coat is a lua value, the desk is a table lua keeps called the registry, and the ticket is a plain int c++ can store anywhere

the map

diagram

the plan, nine sections

1. the coat desk why a value cannot stay, and the registry

2. lua_ref_t the ticket as a class, move only

3. outliving the state a ticket handed back after the desk has burnt down

4. lua_function_t calling lua from c++, with nothing thrown

5. std::function a lua function wherever c++ wants one

6. an event bus listeners, tokens, and a listener removed mid fire

7. lua_table_t reading and writing a lua table

8. containers std::vector and std::map both ways, and the lua_next trap

9. config a settings.lua into a c++ struct

section 1, the coat desk

a value lives on the stack, then it is gone

every value a c function is handed sits on the lua stack, and the stack is cleared when the function returns

so the obvious idea, keep where it was, goes nowhere

const void* g_kept = nullptr;

int l_keep_pointer(lua_State* L)
{
	// the address of the table, for debugging only
	g_kept = lua_topointer(L, 1);
	return 0;
}

lua_topointer gives back the address of the value, lua documents it as useful for debugging and nothing else

the gc does not know c++ exists

local t = setmetatable({}, { __gc = function() print("collected") end })
keep_pointer(t)
t = nil
collectgarbage()
print("after the gc")
collected
after the gc

the __gc ran, the table is gone, and g_kept points at freed memory

the gc never looked at g_kept, it cannot, a c++ variable is just bytes somewhere in your program, lua has no idea they mean anything

reachable, or gone

the gc keeps a value only if it can reach it, starting from a few roots and following every reference

diagram

anything nothing reaches is garbage, so to keep a value alive c++ has to hang it off a root

the registry, a table only c can see

part 1 already used it, luaL_newmetatable keeps every metatable there under its name

it is a normal lua table, found at the fake stack index LUA_REGISTRYINDEX, and no script can reach it, so nothing a mod does can break what you keep in it

and it is a root, so anything in it lives as long as it stays in it

that makes it the coat desk, all c++ needs now is a way to get a fresh ticket number for each coat

luaL_ref and luaL_unref

lua_pushstring(L, "sword");
int ticket = luaL_ref(L, LUA_REGISTRYINDEX);
lua_rawgeti(L, LUA_REGISTRYINDEX, ticket);
lua_pop(L, 1);
luaL_unref(L, LUA_REGISTRYINDEX, ticket);

luaL_ref pops the value on top, stores it in the registry under a free number, and hands you the number

lua_rawgeti pushes registry[ticket] back, as often as you like

luaL_unref frees the slot, the value is unreachable again and the number goes back on a list to be reused

how lua picks the number

5.5 keeps a list of handed back numbers in registry[1], which is why slot 1 is reserved

a fresh ticket is the next free slot past the end, 4 in a new state, bc 2 and 3 hold the globals and the main thread

an unref puts its number at the front of the list

the next ref takes the front of the list before it ever grows the table

so numbers get reused, and that is fine, right up until a number gets handed back twice

section 2, lua_ref_t

the ticket as a class

class lua_ref_t
{
private:
	// empty once the state has closed, section 3
	std::weak_ptr<lua_State>	m_state;

	// the ticket number, LUA_NOREF when there is none
	int							m_ref	= LUA_NOREF;

public:
	lua_ref_t() = default;

	// pops the value on top of the stack and keeps it
	explicit lua_ref_t(lua_State* L) : m_state(lua_host_t::of(L).life()), m_ref(luaL_ref(L, LUA_REGISTRYINDEX)) {}

	~lua_ref_t() { reset(); }
};

raii again, the ref grabs a ticket in its ctor and hands it back in its dtor, so a ticket can never be forgotten

LUA_NOREF a number lua promises is never a real ticket, -2, our empty value, ignore m_state until section 3

the same script, with a ref

int l_keep_ref(lua_State* L)
{
	lua_pushvalue(L, 1);
	g_held.emplace(L);
	return 0;
}
after the gc
collected

lua_pushvalue(L, 1) copies arg 1 to the top, bc the ctor pops what is on top

now the table survives the gc, the registry reaches it, and collected only prints after g_held->reset() hands the ticket back

why it cannot be copied

a ref that could be copied gives two objects holding one ticket, and both dtors hand it back

copyable_ref_t a { L, luaL_ref(L, LUA_REGISTRYINDEX) };
copyable_ref_t b = a;
a and b both hold ticket 4
potion got ticket 4, goblin got ticket 4
ticket 4 now holds goblin

the second unref put 4 on the free list again, the list now points at itself, so the next two refs both got 4

the potion is gone, its ticket opens the goblin, and nothing crashed, the worst kind of bug, a wrong value with no error

deleting the copy

	// a copy would hand the same ticket back twice
	lua_ref_t(const lua_ref_t&)				= delete;
	lua_ref_t& operator=(const lua_ref_t&)	= delete;
errs.cpp(8): error C2280: 'lua_ref_t::lua_ref_t(const lua_ref_t &)': attempting to reference a deleted function
held.hpp(117): note: 'lua_ref_t::lua_ref_t(const lua_ref_t &)': function was explicitly deleted

same move as part 1s host, a quiet runtime bug turned into a compile error, which is always the better trade

moves, one owner at a time

	// the old one is left empty, so only one of us hands the ticket back
	lua_ref_t(lua_ref_t&& _other) noexcept : m_state(std::move(_other.m_state)), m_ref(std::exchange(_other.m_ref, LUA_NOREF)) {}

	lua_ref_t& operator=(lua_ref_t&& _other) noexcept
	{
		if (this != &_other)
		{
			reset();
			m_state	= std::move(_other.m_state);
			m_ref	= std::exchange(_other.m_ref, LUA_NOREF);
		}

		return *this;
	}

a move hands the ticket over, so a ref can still go in a std::vector or come back from a function

std::exchange(x, v) sets x to v and gives back what x was, take the ticket and blank the old one in one go

reset() first in the assignment, the ticket we already held goes back before we take the new one

push and reset

	// pushes the value, nil when there is none
	void push(lua_State* L) const { lua_rawgeti(L, LUA_REGISTRYINDEX, m_ref); }

	// hands the ticket back, safe after the state has closed
	void reset()
	{
		if (const std::shared_ptr<lua_State> state = lock())
			luaL_unref(state.get(), LUA_REGISTRYINDEX, m_ref);

		m_state.reset();
		m_ref = LUA_NOREF;
	}

push on an empty ref reads registry[-2], which is just nil, so it never needs an if

luaL_unref of LUA_NOREF does nothing, the library checks for it, so a moved from ref is safe to destroy

lock is section 3

exercise 1, lua_ref_t

write lua_ref_t without the m_state part, a raw lua_State* is fine for now

pseudo

  1. the ctor calls luaL_ref, the dtor calls reset
  2. delete both copies
  3. both moves, with std::exchange blanking the source
  4. push and reset
  5. test, ref a table with a __gc that prints, move the ref into a std::vector, reset it, and see collected print exactly once

tip keep the copyable_ref_t demo around and run it once, seeing the potion turn into a goblin is what makes the deleted copy stick

section 3, outliving the state

a ticket handed back too late

the desk burns down, and someone walks up with a ticket

std::optional<raw_ref_t> ref;
{
	lua_host_t host(listener);

	lua_pushstring(host.state(), "sword");
	ref.emplace(raw_ref_t { host.state(), luaL_ref(host.state(), LUA_REGISTRYINDEX) });
}

// the host is gone, lua_close has run
ref.reset();

a game has refs everywhere, in components, in the ui, in a timer list, and they get torn down in whatever order the game shuts down in

so a ref will outlive the state sooner or later

what that does

host closing
ref going
Segmentation fault

that last line is git bash naming the windows access violation, the process just dies

and under msvcs /fsanitize=address, with lua built from source so the sanitiser can see inside it

ERROR: AddressSanitizer: heap-use-after-free on address 0x120323ea0628
READ of size 8 at 0x120323ea0628 thread T0
    #3 in luaL_unref lauxlib.c:887
    #4 in raw_ref_t::~raw_ref_t host.cpp:133
freed by thread T0 here:
    #4 in lua_close lstate.c:416

the unref read the registry, and lua_close had already freed it

two honest answers

the host refuses to close while any ref is alive, count refs in the host and assert in its dtor

refs go dead when the state closes, a dead ref skips the unref and pushes nothing

the first costs nothing at runtime, but it turns your shutdown order into a crash, and that order is the hardest thing in a game to keep straight

the second costs a little per ref, and any order just works, so we pick that one, a ref can now be declared anywhere in main, before or after the host

the host holds a shared_ptr

class lua_host_t
{
private:
	// closed when the last owner lets go, and the host is the only owner
	std::shared_ptr<lua_State>	m_state;

	// who hears about errors
	lua_listener_t&				m_listener;

public:
	lua_host_t(lua_listener_t& _listener) : m_state(luaL_newstate(), lua_close), m_listener(_listener) { ... }

	// a ticket that goes dead when the state closes
	std::weak_ptr<lua_State> life() const { return m_state; }
};

shared_ptr(ptr, lua_close) the second arg is the deleter, what to call instead of delete, so the host no longer needs a dtor at all

the host never shares it, it is the only owner, the shared_ptr is there so it can hand out the next thing

weak_ptr, a ticket that knows

a std::weak_ptr points at what a shared_ptr owns without owning it

expired() says whether the owner has let go

lock() gives a shared_ptr while it is still alive, or an empty one once it is gone

so the ref keeps a weak_ptr<lua_State>, and reset only unrefs when lock hands something back

	// the state, null once it is closed, hold it for the length of a call
	std::shared_ptr<lua_State> lock() const { return m_ref == LUA_NOREF ? nullptr : m_state.lock(); }

the shared_ptr from lock keeps the state alive for as long as you hold it, so hold it for one call, never store it

finding the host from L

the ref ctor only gets L, so it needs a way from L back to the host

	// the extra space is one pointer, ours to fill, every coroutine gets a copy
	*static_cast<lua_host_t**>(lua_getextraspace(L)) = this;

	// the host behind any thread of our state
	static lua_host_t& of(lua_State* L) { return **static_cast<lua_host_t**>(lua_getextraspace(L)); }

lua_getextraspace lua reserves the size of one pointer in front of every state, just for the host to use

and a coroutine gets a copy of it, so L from inside a coroutine still finds the host, part 5 leans on that

the late ticket, landed

std::optional<lua_ref_t> ref;
{
	lua_host_t host(listener);

	lua_pushstring(host.state(), "sword");
	ref.emplace(host.state());
}

ref.reset();
host closing
ref going, alive: 0
done

the ref saw the state was gone, skipped the unref, and the program carried on, sanitiser clean

what going dead costs

16 bytes a weak_ptr is two pointers, against 8 for a raw lua_State*

a counter bump per lock lock adds 1 to a shared count and the dtor takes it off, it is an atomic add, a few nanoseconds

one shared block the count lives next to the state, allocated once by the host

next to a lua_pcall that is noise, and it buys a shutdown that cannot crash

one thing it does not buy, threads, two threads touching one state still need a lock, part 5

exercise 2, refs that go dead

pseudo

  1. move the host onto a std::shared_ptr<lua_State> with lua_close as the deleter
  2. life() and of(L), store this in the extra space in the ctor
  3. the ref keeps a weak_ptr, lock and reset as on the slides
  4. run the late reset, then run it again with the ref declared before the host

tip declare the ref first and it is destroyed last, after the host, which is the case that used to crash, if it prints done both ways the order has stopped mattering

section 4, lua_function_t

a function type as a template arg

template <typename sig_t>
class lua_function_t;

template <typename result_t, typename... args_t>
class lua_function_t<result_t(args_t...)>
{
private:
	// the function itself, in the registry
	lua_ref_t	m_ref;

	// ... operator()
};

int(int) is a function type, not a pointer, a function taking an int and giving back an int, the same thing std::function<int(int)> takes

the partial specialisation pulls it apart, result_t is int, args_t... is int, part 2s fn_traits_t trick again

so lua_function_t<void(entity_t&, int)> reads like the lua function it holds

what a call gives back

a lua function can fail, a typo, a nil, an error(), and c++ has to hear about it without an exception flying through the engine

	// true or false for a void function, empty or a value otherwise
	using call_result_t = std::conditional_t<std::is_void_v<result_t>, bool, std::optional<result_t>>;

std::conditional_t<test, a, b> picks a type at compile time, a when the test is true, b when it is not

why not std::optional<void> it does not exist, optional needs a real type inside, so a void function says yes or no with a bool

operator(), the call

	call_result_t operator()(args_t... _args) const
	{
		const std::shared_ptr<lua_State> state = m_ref.lock();
		if (!state)
			return {};

		lua_State* L		= state.get();
		lua_host_t& host	= lua_host_t::of(L);
		const int base		= lua_gettop(L);

		m_ref.push(L);
		(push_result<args_t>(L, std::forward<args_t>(_args), 0), ...);

		if (!host.pcall(sizeof...(args_t), 1))
			return {};

		// ... read the result, next slides
	}

a dead state returns empty, nothing to call

push_result is part 3s, so an entity_t& arg goes out as a borrowed card and an int as a number, the same rules as a return

host.pcall is part 1s, it reports the error to the listener and pops it, it moves from private to public now

reading the result, the trap

part 2 reads a value with check then read, so the obvious thing is

lua_traits_t<int>::check(L, -1);
checking the result
PANIC: unprotected error in call to Lua API (bad argument #-1 (number expected, got string))

and then the program aborts, exit code 3, the game is gone

why it panics

check raises, a failed check is a lua error, a jump to the nearest lua_pcall

inside a wrapper there always is one, the script that called us is running inside the hosts pcall

out here c++ called lua, the pcall has already returned, and there is nothing to land in

so lua calls its panic function, prints that line, and calls abort

the rule, outside a lua call nothing may raise, so c++ needs a way to ask about a value that cannot fail

the fourth question, test

every page grows a test, it answers the same question as check, and never raises

template <lua_integer int_t>
struct lua_traits_t<int_t>
{
	// ... push, check, read

	// never raises, safe outside a lua call
	static bool test(lua_State* L, int _idx)
	{
		int is_int				= 0;
		const lua_Integer value	= lua_tointegerx(L, _idx, &is_int);
		return is_int && std::in_range<int_t>(value);
	}
};

lua_tointegerx reads an integer and sets the flag to say whether it worked, so 3.0 passes and 3.5 does not, the same as luaL_checkinteger

the range check from part 2 carries over, a test that let 2^40 through as an int would be lying

the other pages are one liners, lua_isboolean, lua_isnumber, lua_isstring, the optional one is nil or the inner test

the result, read with test

		if constexpr (std::is_void_v<result_t>)
		{
			lua_settop(L, base);
			return true;
		}
		else
		{
			// check would raise, and out here nothing would catch it
			call_result_t result;
			if (lua_traits_t<result_t>::test(L, -1))
				result = lua_traits_t<result_t>::read(L, -1);
			else
				host.report(std::string("a callback returned the wrong type, got ") + luaL_typename(L, -1));

			lua_settop(L, base);
			return result;
		}

note the top, do the work, reset the top, part 1s habit, base was taken before anything was pushed

three calls, nothing thrown

function double(x) return x * 2 end
function broken(x) return x.y end
function wordy(x) return "lots" end
[lua] script:3: attempt to index a number value (local 'x')
[lua] a callback returned the wrong type, got string
double 42, broken 0, wordy 0

double(21) gives 42, the other two come back empty, and both told the listener why

the caller only ever sees a value or nothing, so a modders typo costs one log line, never the frame

a lua function as an arg

the function has to come from somewhere, usually a script handing it over, so it gets a phrasebook page

template <typename sig_t>
struct lua_traits_t<lua_function_t<sig_t>>
{
	static void push(lua_State* L, const lua_function_t<sig_t>& _value) { _value.push(L); }
	static void check(lua_State* L, int _idx) { luaL_checktype(L, _idx, LUA_TFUNCTION); }
	static bool test(lua_State* L, int _idx) { return lua_isfunction(L, _idx); }

	static lua_function_t<sig_t> read(lua_State* L, int _idx)
	{
		lua_pushvalue(L, _idx);
		return lua_function_t<sig_t>(lua_ref_t(L));
	}
};

so a lua function can be an arg to any generated wrapper, the page is all part 2 needs

and this is how the demo above got its functions, section 7 builds get

auto double_fn = host.globals().get<lua_function_t<int(int)>>("double");

exercise 3, lua_function_t

pseudo

  1. add test to the bool, integer, float, string and optional pages
  2. the class, m_ref and call_result_t
  3. operator(), lock, note the top, push, host.pcall, test, read, reset the top
  4. the page, so a function can arrive as an arg
  5. call double, broken and wordy and match the output above

tip take base before the push, if you take it after, the reset leaves the function behind on every call, and a stack that grows by one per frame is the slow creep part 1s stack guard exists to catch, wrap a test call in a lua_stack_guard_t

section 5, std::function

game code wants std::function

part 1 set the rule, game classes never include lua.h

but a game api that takes a callback has to name some type for it, and the c++ answer is std::function

token_t connect(std::function<void(entity_t&, int)> _fn);

so the phrasebook gets a page for std::function<sig_t>, and a lua function can go anywhere a c++ callback can, the game never knows it was lua

std::function wants copies

the obvious read, wrap the lua_function_t straight in

std::function<void(int)> fn = std::move(held);
functional(1104): error C2338: static_assert failed: 'The target function object type must be copy constructible (N4988 [func.wrap.func.con]/10.1).'
        with
        [
            _Fx=lua_function_t<void (int)>
        ]

std::function can be copied, so whatever it holds must be copyable too, and our ref deleted its copy on purpose in section 2

so the copies share one

	static std::function<result_t(args_t...)> read(lua_State* L, int _idx)
	{
		// std::function copies what it holds, a ref cannot be copied, so the copies share one
		auto held = std::make_shared<held_t>(lua_traits_t<held_t>::read(L, _idx));

		return [held](args_t... _args) -> result_t
		{
			if constexpr (std::is_void_v<result_t>)
				(*held)(std::forward<args_t>(_args)...);
			else
				return (*held)(std::forward<args_t>(_args)...).value_or(result_t{});
		};
	}

the lambda captures a shared_ptr, and copying a shared_ptr is fine, so every copy of the std::function shares one ticket

the ticket goes back when the last copy dies, the same shared ownership as part 3s loot, just on the c++ side

the payoff, a game method taking a callback

// calls the given function once per item
void each(const std::function<void(const std::string&, int)>& _fn) const
{
	for (const auto& [name, count] : m_items)
		_fn(name, count);
}
inv:each(function(name, count) print("each", name, count) end)
each	arrow	20
each	potion	5
each	sword	1

inventory_t still has no lua.h, and method<&inventory_t::each>("each") binds it with no new code, the arg page does the lot

a result c++ insists on

std::function<int(int)> has to give back an int, it cannot say empty

value_or(result_t{}) a failed call gives a default, 0 for an int, "" for a string, and the listener already heard why

that is a real loss, a caller cannot tell 0 from failed, so when it matters, ask for std::function<std::optional<int>(int)>, the optional page makes that just work

c++23 has std::move_only_function, which would hold the ref with no shared_ptr, this series sticks to c++20

exercise 4, the std::function page

pseudo

  1. a partial specialisation on std::function<result_t(args_t...)>
  2. check and test as the function page, read only, no push
  3. read builds a held_t through the function page, into a make_shared
  4. return a lambda capturing it, if constexpr on void

tip capture held by value, [held], a [&held] captures the local, which dies when read returns, and the first call reads freed stack memory

section 6, an event bus

signal_t, the game side

template <typename... args_t>
class signal_t
{
private:
	struct slot_t
	{
		// what disconnect is handed
		token_t							token	= 0;

		// what gets called
		std::function<void(args_t...)>	fn;
	};

	std::vector<slot_t>		m_slots;
	token_t					m_next	= 1;

public:
	token_t connect(std::function<void(args_t...)> _fn);
	void disconnect(token_t _token);
	void fire(args_t... _args);
};

plain c++, no lua.h, the events class is just named signals, signal_t<entity_t&, int> hit;

a token is what connect hands back, a number to say which listener to take off later, 0 is never handed out

fire, on a snapshot

	void fire(args_t... _args)
	{
		// a copy, a listener may connect or disconnect while we walk it
		const std::vector<slot_t> snapshot = m_slots;

		for (const slot_t& slot : snapshot)
		{
			if (connected(slot.token))
				slot.fn(_args...);
		}
	}

a listener is lua code, and lua code can call events.on or events.off in the middle of a fire

walking m_slots itself while it grows means a push_back can move every slot to new memory, mid call, under the function that is running

the copy is cheap, each std::function copy is one shared_ptr bump, and the connected check skips anyone taken off since

events.on, from lua

	static int l_on(lua_State* L)
	{
		events_t& events = from_upvalue(L);

		// pass 1, the name is one we know, and a function came with it
		const int which = luaL_checkoption(L, 1, nullptr, m_names);
		luaL_checktype(L, 2, LUA_TFUNCTION);

		// pass 2, the std::function page builds the listener
		return lua_guard(L, [&]
		{
			token_t token = 0;
			if (which == 0)
				token = events.hit.connect(lua_traits_t<std::function<void(entity_t&, int)>>::read(L, 2));
			// ... the other names

			lua_pushinteger(L, token);
			return 1;
		});
	}

luaL_checkoption matches arg 1 against a null ended list of names and gives back its index, a typo is an error, bad argument #1 to 'on' (invalid option 'hti')

each name has its own signature, so this part is a hand written module, part 1 style, and the hard part, a lua function into a std::function, is one page read

the script, and one broken listener

events.on("hit", function(target, damage) print(target:describe() .. " took " .. damage) end)
events.on("hit", function(target, damage) error("this listener is broken") end)

local once
once = events.on("hit", function(target, damage)
	print("once, then gone")
	events.off("hit", once)
end)

events.on("hit", function(target, damage) print("still here, health " .. target:health()) end)
goblin took 3
[lua] script:2: this listener is broken
once, then gone
still here, health 7

the broken one did not stop the rest, bc the pcall lives inside each lua_function_t, a lua error never leaves its own listener

the second hit

goblin took 2
[lua] script:2: this listener is broken
still here, health 5
listeners left 3

once took itself off during the first fire, so it is gone for the second

and the broken one stays connected, it fails every time and says so every time, whether to drop a listener after n failures is a game decision, not the bus

the flow, c++ fires into lua

diagram

the std::function sits between signal_t and lua_function_t, it only forwards, so it is left off

exercise 5, the event bus

pseudo

  1. signal_t with connect, disconnect and fire on a snapshot
  2. events_t with a hit and a said signal
  3. the module, on with luaL_checkoption, off taking the name and the token
  4. run the four listener script above, and match both fires

tip luaL_checkoption wants the list to end in nullptr, and its third arg is the default when the arg is missing, pass nullptr there too so a missing name is an error instead of quietly meaning hit

section 7, lua_table_t

the books way, and ours

the book reads a table by global name, every call looks the global up again, pushes the key, reads, and checks the type with an assert

that only reaches globals, and a release build compiles the assert out, so a wrong type reads garbage

ours holds a ref, so it reaches any table, a global, a field of another table, an arg, the return of a chunk, and a wrong type is an error that names the key

class lua_table_t
{
private:
	lua_ref_t	m_ref;

public:
	explicit lua_table_t(lua_ref_t _ref) : m_ref(std::move(_ref)) {}

	// ... get, set, operator[], size, push
};

get, test then read

	template <typename value_t>
	value_t get(const lua_key_t& _key) const
	{
		lua_State* L = state();
		lua_stack_guard_t guard(L);

		m_ref.push(L);
		_key.push(L);
		lua_rawget(L, -2);

		if (!lua_traits_t<value_t>::test(L, -1))
		{
			const std::string got = luaL_typename(L, -1);
			lua_pop(L, 2);
			throw std::runtime_error(_key.text() + (got == "nil" ? " is missing" : " is the wrong type, got " + got));
		}

		value_t value = lua_traits_t<value_t>::read(L, -1);
		lua_pop(L, 2);
		return value;
	}

lua_rawget skips metatables, so no __index can run lua code here, and lua code could raise with nothing to land in, section 4s trap again

a throw is fine here we are in plain c++, and inside a wrapper part 1s lua_guard turns it into a lua error

part 1s stack guard checks every path pops what it pushed

a key, a name or an index

class lua_key_t
{
private:
	std::variant<std::string, lua_Integer>	m_key;

public:
	lua_key_t(const char* _name) : m_key(std::string(_name)) {}
	lua_key_t(int _index) : m_key(static_cast<lua_Integer>(_index)) {}

	// ... push, and text for the error message
};

a converting ctor per kind, so get<int>("hp") and get<std::string>(1) both just work

set is get backwards, push the table, the key, the value, lua_rawset, and pop the table

reading the player

player = { name = "finn", hp = 20, bag = { "sword", "potion" } }
lua_table_t player = host.globals().get<lua_table_t>("player");

player.get<std::string>("name");		// finn
player.get<int>("mana");				// throws, mana is missing
player.get<int>("name");				// throws, name is the wrong type, got string

host.globals() the globals are a table too, lua_pushglobaltable then a ref, so a global function or table is just a get

and a table inside a table comes out with get<lua_table_t>, the table class has a page of its own, the same three lines as the function page

operator[], a proxy

player["hp"] = 15;
int hp = player["hp"].as<int>();
lua_table_t bag = player["bag"].as<lua_table_t>();
class lua_proxy_t
{
private:
	// the table, the proxy must not outlive it
	const lua_table_t&	m_table;

	// which entry
	lua_key_t			m_key;

public:
	template <typename value_t>
	lua_proxy_t& operator=(const value_t& _value) { m_table.set(m_key, _value); return *this; }

	template <typename value_t>
	value_t as() const { return m_table.get<value_t>(m_key); }
};

player["hp"] does nothing to lua, it just remembers which table and which key

an = turns into a set, an .as<int>() into a get, so one [] both reads and writes, which a plain return value never could

the proxy trap

the proxy holds a reference to its table, so it must not outlive it

auto hp = host.globals()["player"];

host.globals() is a temporary, it dies at the ;, and hp now refers to a table that is gone

so a proxy is for one line, t["x"] = 1, or t["x"].as<int>(), never stored with auto

the standard library has the same shape, std::vector<bool> hands back a proxy from [] for the same reason, and the same warning comes with it

a method on a lua table

the book calls table functions with or without self, and with lua_function_t that is no special case

bag = { items = {} }
function bag:add(name) self.items[#self.items + 1] = name end
function bag:count() return #self.items end
// a method is a plain function, self is just the first arg
auto add	= bag.get<lua_function_t<void(const lua_table_t&, std::string)>>("add");
auto count	= bag.get<lua_function_t<int(const lua_table_t&)>>("count");

add(bag, "sword");
add(bag, "potion");
count 2

function bag:add(name) is sugar for function bag.add(self, name), so c++ passes the table as arg 1 and that is the : call

exercise 6, the proxy

pseudo

  1. lua_key_t with a converting ctor per kind, push and text
  2. set on the table
  3. lua_proxy_t, a reference to the table and a key, operator= and as
  4. lua_table_t::operator[] returns one
  5. player["hp"] = 15 from c++, then print(player.hp) from lua says 15

tip operator[] returns a proxy by value, so the proxy has to be a complete type where operator[] is defined, declare it in the table, write the proxy under the table, then define operator[] under the proxy

section 8, containers

part 2s refusal, lifted

part 2 refused to bind inventory_t::list(), it returns a std::vector<std::string> and there was no page

	static void push(lua_State* L, const std::vector<value_t>& _value)
	{
		lua_createtable(L, static_cast<int>(_value.size()), 0);

		for (size_t i = 0; i < _value.size(); i++)
		{
			lua_traits_t<value_t>::push(L, _value[i]);
			lua_rawseti(L, -2, static_cast<lua_Integer>(i + 1));
		}
	}

lua_createtable(L, n, 0) a table with room for n array slots already, no regrowing as it fills

i + 1 c++ counts from 0, lua from 1, this is the one line where that bites

method<&inventory_t::list>("list") now binds, and ipairs(inv:list()) prints 1 potion, 2 sword

reading one back

check it is a table, then every item, 1 to lua_rawlen, through the item pages test

read the same loop, lua_rawgeti each item and the item pages read

			lua_rawgeti(L, _idx, i);
			if (!lua_traits_t<value_t>::test(L, -1))
				luaL_argerror(L, _idx, lua_pushfstring(L, "item [%I] is a %s", i, luaL_typename(L, -1)));

test again, not the items check, an items check blames its stack slot, a probe doing exactly that got bad extra argument #-1 (string expected, got boolean), a number that means nothing to the script

so the vector reports against its own arg, and names the item

script:1: bad argument #1 to 'add_names' (item [3] is a boolean)

walking a table, lua_next

a map has no 1 to n, its keys are anything, so it gets walked with lua_next

lua_pushnil(L);
while (lua_next(L, _idx))
{
	// key at -2, value at -1
	// ... use them
	lua_pop(L, 1);
}

start with nil on top, it means from the beginning

each step pops the old key and pushes the next key and its value, 0 back when there are no more

pop only the value, the key has to stay on top, it is how lua_next knows where it got to

the key trap

	while (lua_next(L, 1))
	{
		std::cout << "key " << lua_tostring(L, -2) << "\n";
		lua_pop(L, 1);
	}
keys_in_place({ "sword", "potion", gold = 50 })
key 1
[lua] invalid key to 'next'

lua_tostring on a number does not just read it, it changes the slot into the string "1"

then lua_next gets "1" back as the key, the table has no key "1", only 1, and it gives up

the fix, read a copy

	lua_pushnil(L);
	while (lua_next(L, _idx))
	{
		// key at -2, value at -1, read the key from a copy, never in place
		lua_pushvalue(L, -2);
		const bool fits = lua_traits_t<key_t>::test(L, -1) && lua_traits_t<value_t>::test(L, -2);

		if (!fits || !_fn(-1, -2))
		{
			lua_pop(L, 3);
			return false;
		}

		lua_pop(L, 2);
	}

lua_pushvalue(L, -2) copies the key to the top, the string page can turn the copy into whatever it likes

this one loop, lua_each_pair, is shared by the map pages test and read, so the trap is fixed once

one page, two maps

template <typename key_t, typename value_t>
struct lua_traits_t<std::map<key_t, value_t>> : lua_map_traits_t<std::map<key_t, value_t>> {};

template <typename key_t, typename value_t>
struct lua_traits_t<std::unordered_map<key_t, value_t>> : lua_map_traits_t<std::unordered_map<key_t, value_t>> {};

lua_map_traits_t<map_t> reads map_t::key_type and map_t::mapped_type, so it does not care which map it is, the same inherit trick as part 2s lua_object_traits_t

local counts = inv:counts()
print(counts.potion, counts.sword)
inv:add_all({ arrow = 20, potion = 2 })
3	1

a copy each way, on purpose

local names = inv:list()
names[1] = "axe"
print(inv:list()[1])
arrow

the table is a copy, changing it changes nothing in c++

plain data names, counts, settings, a copy is exactly right, nothing to keep in sync, nothing to dangle

objects a std::vector<entity_t> would copy every entity, and a script hitting one hits the copy, so objects go out one at a time through part 3s cards, never as a container

exercise 7, containers

pseudo

  1. the vector page, push, then check, test and read over 1 to lua_rawlen
  2. lua_each_pair, the key copy, return false on the first mismatch
  3. lua_map_traits_t, push with lua_rawset, test and read through lua_each_pair
  4. the two one line pages for std::map and std::unordered_map
  5. bind list, counts and add_all, and run the script above

tip lua_rawlen stops being meaningful the moment a table has a nil in the middle, { 1, nil, 3 } can say 1 or 3, so a vector check that walks 1 to rawlen is the honest limit, document that a sequence with holes is not a vector

section 9, config

settings.lua

return {
	title		= "dungeon",
	width		= 1280,
	height		= 720,
	fullscreen	= false,
}
struct settings_t
{
	int						width		= 0;
	int						height		= 0;
	bool					fullscreen	= false;
	std::string				title;
	std::optional<float>	volume;
};

a config file that is lua, so it can have comments, maths, width = 640 * 2, and it is the payoff of this whole part

bare struct fields, no m_, the finn-style rule for a plain struct, it has no rules to keep, just data

a chunk is a function

	// a whole file as a function, empty when it does not compile
	template <typename sig_t>
	std::optional<lua_function_t<sig_t>> lua_host_t::load_file(const std::string& _path)
	{
		if (luaL_loadfile(state(), _path.c_str()) != LUA_OK)
		{
			report();
			return std::nullopt;
		}

		return lua_function_t<sig_t>(lua_ref_t(state()));
	}

part 1 said it, load compiles a file into a chunk, one function, and pushes it

so load and run are two steps again, and a file that returns a table is just lua_function_t<lua_table_t()>

auto chunk = host.load_file<lua_table_t()>("settings.lua");
std::optional<lua_table_t> table = (*chunk)();

read_settings

settings_t read_settings(const lua_table_t& _table)
{
	return
	{
		.width		= _table.get<int>("width"),
		.height		= _table.get<int>("height"),
		.fullscreen	= _table.get<bool>("fullscreen"),
		.title		= _table.get<std::string>("title"),
		.volume		= _table.get<std::optional<float>>("volume"),
	};
}

designated initialisers each .field = names what it fills, the c++20 feature part 1 used for defs

std::optional<float> part 2s optional page, its test passes on nil, so a missing volume is just empty, no special case anywhere

three files

dungeon 1280x720 fullscreen 0 volume -1
settings_missing.lua: height is missing
settings_wrong.lua: fullscreen is the wrong type, got string

a modder who typos fullscrene or writes "yes" gets told which key and what was wrong, not a crash and not a silent false

and the volume printed -1 bc the file left it out, value_or(-1.0f) in the print, the struct holds an empty optional

exercise 8, the config reader

pseudo

  1. host.globals() and host.load_file<sig_t>
  2. settings_t and read_settings, one get per field
  3. main, load, call, read, inside a try, print the message on a throw
  4. run it on the three files above

tip a .title = ... list has to follow the order the fields are declared in, msvc says error C7560: 'width': designators must appear in member declaration order of class 'settings_t', and a braced list runs top to bottom, so the first bad key is the one reported

the shape of it all

diagram

who owns what, part 4 edition

the host owns the state, the only shared_ptr, its death closes lua

a ref owns one registry slot, never the state, it goes dead when the state does

lua_function_t and lua_table_t own one ref each, move only like it

a std::function from lua shares one lua_function_t between all its copies

a container is a copy, each side owns its own

why its built this way

a ref, not a raw int raii hands the ticket back, and the deleted copy makes a double unref impossible

dead refs over a strict shutdown any order in main is safe, for 8 bytes and an atomic bump per call

test beside check outside a lua call nothing may raise, so every page can now answer without raising

errors as empty results a modders bug is a log line, never an exception through the frame

copies for containers plain data has nothing to keep in sync, objects stay cards from part 3

recap

  • the gc only keeps what it can reach from lua, so c++ keeps a value by putting it in the registry and holding the ticket
  • lua_ref_t is that ticket as a move only class, and it goes dead with the state through a weak_ptr
  • lua_function_t<int(int)> calls lua with a pcall, reads the result with test, and gives back an empty optional on any failure
  • a lua function becomes a std::function by sharing one held function between every copy
  • an event bus fires on a snapshot, and each listener fails alone
  • lua_table_t reads and writes any table, [] is a proxy, lua_next reads a copy of the key, containers are copied each way

have a go, the capstone

a timer_system_t in c++ that scripts schedule callbacks on

local t = timers.after(2.0, function() print("two seconds") end)
timers.every(0.5, function() print("tick") end)
timers.cancel(t)
  • after and every hand back a token, cancel takes one off
  • the game calls timers.update(dt) once a frame, and anything due fires
  • a callback that errors is reported and the timers after it still fire
  • a callback that calls cancel or after while being fired is safe

tip keep the timer list game side, a std::function<void()> per timer, the same snapshot as signal_t, and the binding side is one l_after and one l_every, the lua function crosses with the page from section 5

your score

a first try is worth 1, each retry takes 0.2 off so a fifth try is worth 0.2, and a revealed answer is worth 0, tap a question to go back to it

part 5, the host grows up

c++ can now hold lua, and lua can hold c++, both ways safely

part 5 is about the scripts themselves, a sandbox per mod with its own globals, a memory cap, a limit on how long a script may run, and coroutines the frame loop resumes, so wait(2.0) reads like straight line code

the extra space trick from section 3 comes back there, every coroutine finds the host through it