VLC media player has been using Qt interface for quite long time. VLC however, has an option to override window style, which will also change the file dialog as well.
In VLC media player, do the following steps:
Go to Tools > Preferences (or press Ctrl+P)
In the first tab, under Interface Settings - Look and feel, look for "Force window style:" with the drop-down menu and change selection from
System's default to
Finally, click Save to apply the changes.
Then, go to Media > Open File... (or press Ctrl+O) to confirm that the file dialog has been applied with GTK+ window style. That's all.
Tested with VLC 2.2.1 in Debian 8 Xfce (Xfce 4.10).
Force style for Qt5 in Debian/Ubuntu
libqt5libqgtk2 package from the repository, which is available for the following releases of Debian and Ubuntu. No further configuration is needed.
Tested with VLC 2.2.2 in Xubuntu 16.04 (Xfce 4.12). I did not test in Debian, but reportedly works according to this post on Ask Ubuntu.
Force style for Qt5 in other distributions
The package above is not available in repositories of other distributions, including openSUSE, according to this search result from software.opensuse.org. As an alternative, this Arch Wiki noted that
QT_STYLE_OVERRIDE environment variable will force specific style to Qt applications.
Therefore, the line
QT_STYLE_OVERRIDE=GTK+ may be added in one of the following locations:
Without installing the Qt5 package, I have tried export the line to each of above configuration files one at a time, except for the last one. However, none of these worked for VLC in Xubuntu 16.04.
At the moment, I can't verify whether the environment variable actually works or not.