Hi!
I’ve purged all caches, but the problem persists. I’ve also disabled W3TC, so that’s not the cause.
One thing to note, I mistakenly thought the button wasn’t visible on the Contact page, but I was looking at too small a screen.
So right now on all pages the button is visible and the setting doesn’t change this behavior (disabled W3TC and emptied my own browser cache of course.)
I could use display: none via the Customizer, but I only want to do that if I know the normal solution doesn’t work.