Classic Invert reverses the colors of the display. But there's also Smart Invert. To support it, for avoiding images and media from being inverted, you just have to set accessibilityIgnoresInvertColors to true, for these elements. A quick win!

Three examples of the Books app showing the Steve Jobs book by Walter Isaacson. The first one is the default mode. The second one shows all the colors inverted with Classic Invert. The third one shows the colors inverted for everything but the image for the book cover. Some code shows how you just need to set accessibilityIgnoresInvertColors to true for the book cover image.