The Floor That Wasn't There

TLDR: An AI companion was occasionally asking where it had been standing on a floor that no longer existed. Two missing UPROPERTYs, one in AIModule and one in Mover - plus a workaround you can drop in today.

It's funny, really, how the crashes that cost the most time are rarely the dramatic ones. A crash on every run is almost a gift - you catch it in the debugger, fix it, and go and make a cup of tea. It's the one-in-twenty crashes that eat your week. You run it again .. fine. Again .. fine. You add some logging to narrow it down, and now it's fine forever ;p

This was one of those. And, as is so often the case with Unreal, the fix itself turned out to be a couple of lines - finding them was the hard part.

The Crash #

During a stability pass on a UE 5.8 project, one of our automated route runs fell over with this:-

EXCEPTION_ACCESS_VIOLATION
FBasedPosition::operator*()
UPathFollowingComponent::IsBlocked()
UPathFollowingComponent::UpdatePathSegment()
UPathFollowingComponent::TickComponent()
FActorComponentTickFunction::ExecuteTick()
...
UWorld::Tick()

About one run in twenty. Always at the same streaming transition, just after a few sublevels had unloaded. The game has an AI companion that follows the player around using the Mover plugin and a plain AAIController.

A crash in engine code that only turns up one time in twenty usually means something is reading memory that's already been freed. Freed memory tends to hold its old contents for a while, so the read mostly "works" .. until the allocator hands that block to someone else.

What IsBlocked() Reads #

UPathFollowingComponent has a simple stuck detector. Every half a second while the AI follows a path, it records where the pawn's feet are and keeps the last ten positions. IsBlocked() then checks whether all ten are bunched up in one spot.

The samples live in PathFollowingComponent.h:-

	/** location samples for stuck detection */
	TArray<FBasedPosition> LocationSamples;

And an FBasedPosition is a position stored relative to an actor, so that a pawn on a moving platform gets sensible samples:-

USTRUCT(BlueprintType)
struct FBasedPosition
{
	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category=BasedPosition)
	TObjectPtr<class AActor> Base;

	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category=BasedPosition)
	FVector Position;
	...
	ENGINE_API FVector operator*() const;   // Base's transform applied to Position
};

operator* turns a sample back into a world position by reading the base actor's transform. If Base has been freed, that's where we blow up.

Why Garbage Collection Doesn't Help #

Unreal's garbage collector finds references through reflection - UPROPERTY members, plus anything reported from AddReferencedObjects. Base is a UPROPERTY, but that only counts if the struct itself is reached through a property. LocationSamples isn't one. As far as GC is concerned, the array doesn't exist - so it neither keeps the base actors alive nor clears the pointers when they die.

Most of the time that's harmless, as FBasedPosition::Set only stores a base if it isn't Static, and most floors are. The dangerous case is an AI standing on something Movable in a level that gets streamed out:-

1. The AI walks across a Movable mesh in a streamed sublevel. All ten samples name that mesh as their base.
2. The sublevel streams out. Its actors are marked as garbage, and the next GC frees them (s.ForceGCAfterLevelStreamedOut=1 runs that GC straight away).
3. The AI is still following its path. Within half a second it takes a new sample, and IsBlocked() reads the other nine.

In real-life terms: you've written yourself directions - "three doors past the Red Lion". Overnight the pub is knocked down and a block of flats goes up in its place. The council updates its maps, because the pub was on its register. Your note was never on anyone's register, so nobody corrects it - and the next time you follow it, you walk into a stranger's flat.

That's the whole bug. The directions are the samples, each relative to a base actor. The council's register is UPROPERTY - GC fixes up every reference it knows about. The note in your pocket is LocationSamples. And the stranger's flat is whatever the allocator put in that memory next, which is why it only crashes some of the time.

Mover, Not Character Movement #

Where the base comes from depends on the movement component's GetFeetLocationBased():-

• Mover (UNavMoverComponent) fills it from LastFoundDynamicMovementBase, which is set whenever the pawn walks on a Movable primitive. Mover's own copy is a TWeakObjectPtr - safe. The copy handed to path following is a plain pointer - not safe.
• Character Movement doesn't override it, so its samples never have a base at all.

So in 5.8 this is a Mover problem in practice, even though the bug is in AIModule.

Making It Crash Every Time #

One crash in twenty runs is a lousy test. Two things fixed that.

Check addresses, don't dereference them. Before a level is removed (FWorldDelegates::PreLevelRemovedFromWorld) I record the address of every actor in it along with a TWeakObjectPtr. After GC (FCoreUObjectDelegates::GetPostGarbageCollect) I compare every sample's base against that list. Any address whose weak pointer is now invalid is a dangling sample. In the original project this found 4 of them on the companion, right after the GC that followed the level unload.

Make freed memory unreadable. Unreal has allocators for exactly this:-

• -stompmalloc (non-shipping game builds) puts every allocation on its own pages and decommits them on free, so any stale read faults immediately.
• -stomp2malloc -MallocStomp2MinSize=N -MallocStomp2MaxSize=M does the same in the editor, limited to a size range. Restrict it to the size of the platform actor and the editor stays usable.

With either one, a small repro (static ground, a sublevel with one Movable cube, an AI pawn walking across it, unload, force GC) crashes on the first IsBlocked() after the GC. Every time.

Handy tip if you're chasing an engine crash on a launcher build: the editor DLLs ship without symbols, so editor callstacks only show module names. A packaged Development build of a small repro does have full symbols, because the project's PDB includes the engine code.

A Second One, In Mover #

The repro turned up a second crash with the same root cause. With GC on the very next frame after the unload, the Mover pawn sometimes died inside Mover before path following got a look in:-

FWeakObjectPtr::operator=()
FTickFunction::RemovePrerequisite()
UBasedMovementUtils::RemoveTickDependency()
UMoverComponent::OnPostSimulate()
UMoverNetworkPredictionLiaisonComponent::SimulationTick()

UMoverComponent remembers the component it added a tick dependency on, so it can remove it later:-

	TObjectPtr<UPrimitiveComponent> MovementBaseDependency;	// used internally for based movement scheduling management

No UPROPERTY there either. If GC frees the platform before Mover has run a simulation step, the next step calls RemoveTickDependency() on freed memory. Whether a step sneaks in between depends on frame timing - which is why this one comes and goes.

The Fix #

For both, the engine fix is simply to let GC see the references:-

	UPROPERTY(Transient)
	FBasedPosition CurrentDestination;      // UPathFollowingComponent, same pattern

	UPROPERTY(Transient)
	TArray<FBasedPosition> LocationSamples; // UPathFollowingComponent

	UPROPERTY(Transient)
	TObjectPtr<UPrimitiveComponent> MovementBaseDependency; // UMoverComponent

Streamed-out actors are marked as garbage, and with garbage elimination (on by default) GC nulls references to garbage objects rather than leaving them dangling. A sample whose base was destroyed just has no base any more. Worst case, one slightly wrong stuck-detection result.

Here's how the repro runs came out (packaged Development, -stompmalloc):-

A Workaround You Can Ship Today #

You can't rebuild a launcher engine, so here's a world subsystem that empties every path follower's samples just before the GC that follows a level removal:-

void UPathFollowingGuard::Initialize(FSubsystemCollectionBase& Collection)
{
	Super::Initialize(Collection);
	PreLevelRemovedHandle = FWorldDelegates::PreLevelRemovedFromWorld.AddUObject(this, &UPathFollowingGuard::OnPreLevelRemoved);
	PreGCHandle = FCoreUObjectDelegates::GetPreGarbageCollectDelegate().AddUObject(this, &UPathFollowingGuard::OnPreGarbageCollect);
}

void UPathFollowingGuard::OnPreLevelRemoved(ULevel* Level, UWorld* World)
{
	if (Level && World == GetWorld())
	{
		bLevelRemovedSinceLastGC = true;
	}
}

void UPathFollowingGuard::OnPreGarbageCollect()
{
	if (!bLevelRemovedSinceLastGC)
	{
		return;
	}
	bLevelRemovedSinceLastGC = false;
	for (TObjectIterator<UPathFollowingComponent> It; It; ++It)
	{
		if (IsValid(*It) && It->GetWorld() == GetWorld())
		{
			// Keeps block detection on or off as it was; empties LocationSamples.
			It->SetBlockDetectionState(It->IsBlockDetectionActive());
		}
	}
}

It runs at GC time rather than at removal time because the removed level's actors stay valid until the GC - the last moment before they're freed is the safest place to drop references to them. The cost is a few seconds of stuck-detection history after each unload. In the original project it took the dangling-sample count from 4 to 0 across a full playthrough route.

It doesn't cover the Mover crash, as MovementBaseDependency is internal to Mover. If you use Mover with level streaming, a UCharacterMoverComponent subclass (set on the pawn with SetDefaultSubobjectClass) can report it to GC itself:-

void UGCAwareCharacterMoverComponent::AddReferencedObjects(UObject* InThis, FReferenceCollector& Collector)
{
	Super::AddReferencedObjects(InThis, Collector);
	UGCAwareCharacterMoverComponent* This = CastChecked<UGCAwareCharacterMoverComponent>(InThis);
	Collector.AddReferencedObject(This->MovementBaseDependency, This);
}

Takeaways #

• Any TObjectPtr or raw UObject* without UPROPERTY, in a class without AddReferencedObjects, is invisible to GC. That includes pointers tucked inside structs in non-property containers.
• Level streaming is where these surface - it destroys a lot of actors while everything else carries on.
• To pin down a use-after-free, compare addresses instead of dereferencing them, then run with -stompmalloc (game) or -stomp2malloc (editor) to turn "sometimes" into "every time".

• Previous: Is Remote Working The Future?