Custom permalink parameter isn't enforced

I'm trying to create a custom permalink for my posts based on a custom field. I'm most of the way there but I'm struggling with the last part.

My goal permalink structure is as follows:

/%category%/%postname%/%location%

%location% will be replaced with the value of the custom field on my post which I added using the add_meta_box hook. I'm using the post_type_link and post_linkfilters to parse the permalink.

So far, so good. When I open up my home page I can see the posts are being generated with the correct permalinks. Clicking on any of the post links takes me to the correct post page too.

Here is where my problem is. The %location% part of the permalink is completely ignored. I can remove this from the URL or enter an incorrect value but the post is still displayed. So long as the /%category%/%postname% part is correct Wordpress will always show the post page.

This isn't how it should work. If the %location% segment is either missing or doesn't match the value of the location field on the post then it should 404.

How can I get Wordpress to enforce my custom structure when displaying the post page?

Thanks