I mean… just gonna go out on a limb here and guess that the expression exists because within the history of every book ever released, there would be some fantastic books with absolute stinkin’ covers.

Yes, covers are there to help you judge a book. But if that was the only metric you used, you would miss out on some good reads.