The trick to clear the x is to scroll up and click on the color palette.
Here's a timelapse for the curious. https://www.youtube.com/watch?v=G0sG08jKCTI