Monsters generally try to close in on their target via the shortest possible route. If the direct route to the target is blocked by a wall or a solid thing that the monster walks into, it randomly picks another direction to move in for a short while (to find another route to its target) until it reverts to heading straight toward its target again.
A monster does not need to have a line of sight to follow its target, and will attempt to close in on its position within the level.
If a monster with no melee attack reaches its target and "bumps" into it, it will move away from it for a while in a similar fashion it does when hitting any other obstacle.
If a monster has no target, it is in the dormant state and will not move.
A monster chases one target at a time. The monster's initial target is usually the player who awakened the monster from its dormant state by moving into its line of sight, by making an attack in an area the monster can hear (unless the monster has the "deaf" flag set) or by harming the monster. If the initial target was another monster and the monster has not perceived a player while fighting the other monster, it returns to the dormant state after killing its target monster.
After a monster awakens from its dormant state and it is hit for the first time, a target countdown timer called "threshold" is activated. The longer a monster walks around with the same target, the lower the threshold gets. As long as the threshold remains positive, the monster will not change its target even if it is hit by another player or a monster. The monster will only choose a new target if the threshold reaches zero and the monster is hit by another monster or player, or if its current target dies. The threshold system does not apply to the arch-vile, which changes its target immediately to the player or monster that hurts it.
Monsters will only choose another monster as their target if it accidentally hits them with an attack. Arch-viles can never be the target of another monster, nor can pain elementals, as the lost souls launched by a pain elemental are targeted instead.
Ranged attack use
The further a monster is from its target, the more it will use its ranged attack (if it has one), and the closer, the less. The logic behind this is, that when the monster is in close proximity of its target, it will try to get to its target faster to be able to hit it with its melee attack. This does not seem to apply to non-melee monsters such as zombiemen; they will continue executing ranged attacks regularly at close range.