A node builder is a WAD authoring tool.
The VERTEXES, LINEDEFS, SIDEDEFS, and SECTORS lumps fully describe the structure of a map, and it is relatively straightforward for a level editor to create them. However, for rapid rendering and game simulation, the Doom engine needs information on how these structures interrelate. Most time-consuming is the creation of a binary space partition of the map.
Using the basic lumps, a node builder creates the NODES, SEGS, SSECTORS, REJECT and BLOCKMAP lumps (and will likely also add to the VERTEXES lump). Having these structures precalculated and stored in the WAD file, rather than determining them at run time, is key to the speed of the engine.
The node builder used by id Software is called idBSP.
The process of node building is described in some detail in the Unofficial Doom Specs. As detailed in that document, many different partitions of a level are possible, because the choice of partition lines is somewhat arbitrary. There are two competing goals when creating the BSP tree: to have a balanced tree (having about the same number of subnodes on each side) and to minimize the number of splits (breaking linedefs into multiple segs). Thus, different node builders will likely produce different results for a given level.
There exist several different node formats.
The nodes described in the NODES, SEGS and SSECTORS lumps, used by the original version of all Doom-engine games. They suffer from certain limitations, such as a limit of 32768 (or 65536) for the number of segs and vertices and a lack of precision for node coordinates (they are stored as integers, rather than fixed-point values) which results in many slime trails.
To remove the limits of regular nodes, the DeePBSP nodebuilder uses a different format which can be recognized by a starting signature of "xNd4\0\0\0\0" (0x784E643400000000) in the NODES lump. DeeP nodes use 32-bit signed values to reference vertices and segments, pushing the limits from 64K to 2G. However, they still suffer from lack of precision. DeeP nodes are supported by Risen3D, as well as PrBoom+ 126.96.36.199+, ZDoom 2.5.0+ and GZDoom 1.5.0+.
OpenGL renderers have additional requirements for the nodes that the software renderers can ignore, as they are much stricter on their need for subsectors to be convex and precision of coordinates is paramount. The glBSP nodebuilder provides a standard for GL nodes, which are stored in additional lumps so as not to conflict with regular nodes: GL_VERT, GL_SEGS, GL_SSECT, GL_NODES and GL_PVS. These lumps may be in a separate file from their associated level, allowing the distribution of "GWA" files which contains prebuilt GL nodes for commercial levels without having to distribute modified versions of the levels themselves. To match the nodes with a level, a GL_LEVEL lump is used, generally containing a checksum of the level data to make sure the nodes are attributed properly. There are five different versions of GL nodes. Versions 1, 3 and 4 are considered deprecated now. Versions 2 is the preferred format, with version 5 used for maps requiring higher limits. GL nodes are supported by all OpenGL ports, though the extent of the support may vary, since the GL_PVS lump is considered optional and only V2 and V5 are required to be supported for an implementation to be considered valid.
ZDoom extended nodes
ZDoom's extended nodes, built by ZDBSP, use 32-bit unsigned values to refer to vertices, segments and subsectors, pushing the limit to 4G for each. In addition, all structures are listed in the same lump (NODES), including additional vertices created by the nodebuilder. The SEGS and SSECTORS lumps are left empty. ZDoom extended nodes can be compressed or not; if uncompressed they start with an "XNOD" signature (0x584E4F44), if compressed with a "ZNOD" one (0x5A4E4F44). ZDBSP also offers extended GL nodes. They are also stored in a single lump (the SSECTORS lump in a Doom- or Hexen-format map, the ZNODES lump in UDMF) and use a starting signature of XGLN or XGL2 for uncompressed, ZGLN and ZGL2 for compressed. XGLN/ZGLN is used for normal maps, XGL2/ZGL2 is used for maps with more than 65536 linedefs (which are only possible in UDMF). ZDoom compressed nodes are supported by ZDoom and all derivative engines. Uncompressed extended nodes, a more recent addition, are so far supported by ZDoom 2.5.0+, GZDoom 1.5.0+, PrBoom+ 188.8.131.52+ and Eternity Engine 3.39.20+.