Lua stuff is pretty easy, once you get the hang of it.
For implementing Lua hooks, see: SetHookVar, SetHookObject, RemHookVar, RunCondition(?) and IsConditionOverride. These handle setting hook variables (SetHookVar), or object handles (SetHookObject), and removing them after the hook has executed (RemHookVar).
RunCondition is a bit more complex. It takes one argument, an object pointer. This is used to evaluate the preceding 'conditional' fields in scripting. The "condition" variable is used for a CHA_ define from scripting.h, which determines which action this is. You will need to define an action; do a global search for one of the CHA_ defines and it should be pretty clear, you will probably want to familiarize yourself with the conditional scripting .tbl syntax. I made a general overview of it in my pre-commit build thread, in the Recent Builds forum.
IsConditionOverride is for determining whether the original code functionality will be preserved. There is an order note here - things should always be defined in the order
1) if(!IsConditionOverride)
2) do_regular_stuff();
3) else RunCondition();
This allows for the override condition to do any setup for the RunCondition function, as well as evaluate things in their pre- state to toggle the regular stuff. Having RunCondition() (or any other scripting hook) after the default behavior allows the modder to selectively override the parts of the behavior that they want, if they don't want to rewrite that entire chunk of code. They can also use the override hook to gather any data required for that.
For implementing new Lua functions and libraries, see: ade_lib, ade_obj, ADE_FUNC, ADE_VIRTVAR, and ADE_INDEXER. ade_lib defines a new library, or, if the parent field is specified, a sub-library. An example of this is the Ships array in the Mission library. ade_obj defines a new object type, it is a template that takes the data storage type for the object in question (For example, an image handle would be <int>). You may want to use your own helper structs to allow you to check if the passed handle or object is valid; I've tried to keep Lua as safe as possible, so that only critical errors will cause it to completely crash.
ADE_FUNC defines a basic function, and uses the ade_get_args and ade_set_args to parse and return arguments, respectively. I recommend you take a look at existing functions to get the hang of using these. You can also use it to define operators, there is an array at the top of lua.cpp that lists all the operators. Please do not specify __index or __newindex unless you know exactly what you are doing; teh ADE_INDEXER define will probably do what you want.
ADE_INDEXER is essentially identical to ADE_FUNC, and is generally used for array usage ([]). See existing examples. ADE_SETTING_VAR is true when the form is Object[key] = something, ie the array index is being set rather than read.
ADE_VIRTVAR defines a function that will appear as a variable from within Lua. It's much like ADE_INDEXER, in that it uses ADE_SETTING_VAR and the basic function _get and _set functions. Note that both ADE_VIRTVAR and ADE_INDEXER should always return the appropriate value, even when the variable is set to.
Conventions: All Lua libraries and objects are l_<NAME> in lua.cpp. All Lua functions use Java-style naming, so getThis() or setThat() or doThis(). All functions that return handles, should always return handles. If there's a chance of failure, it should return an invalid handle, so that the Lua interpreter doesn't explode when a boolean or nil value is indexed. In addition, handles should define an isValid() function that checks whether the handle is both valid, and if the object is still alive (if applicable). Basically, this is so that a modder has a fast easy way of checking whether they can safely do something with a handle. All Lua variables (and libraries) use the general C++-style naming conventions, Variable, OtherVariable, MyArray, and so on. And all shorthand library names are two lower-case letters, preferably close together on a QWERTY keyboard and that are relatively easy to remember. (Basically: use common sense.

)
References: Nothing official and up-to-date, unfortunately. -output_scripting will output all declared Lua functions, libraries, objects, and so on. This is generated from the description fields of the macros/functions.
Disclaimer: All this is generally based on What Made The Most Sense To Me The Last Time I Worked On It. Some of it may be inconsistent, owing to the gradual year-plus development of everything. There's also quite a bit of stuff I haven't mentioned because I hope that it's pretty self-explanatory from the way it's used in the code, if you have questions, go ahead and ask, I'd just rather not spend a couple hours writing everything up for a forum post that will more than likely be forgotten & ignored once this thread drops off the first page. :-/ And I've tried to comment things decently, but there are some sections that can just be pretty damn confusing, even if you know exactly what's going on. I've tried to keep their sections in their own little world, that nobody needs to mess with, except to make serious lowlevel changes to the system. You shouldn't _ever_ need to change a function with more than 3 "lua_" functions grouped together, unless you're fixing a bug with the scripting system itself.