Sure, just add those extensions you'd like to see in a Hover Box to Previewed Formats the Text Category. Click on Tools > Configuration > Preview > Reviewed Formats. Click on the Text file category in the upper Categories box. Then click on "Add..." under the Category: Text Files box below. Add the extensions you want to see in Hover Box. Separate multiple extensions with a ";" semicolon. Click on apply and OK.
It should actually work out of the box. But only for files with at least one CRLF line break. (A primitive test for a text file.) What is in your files?