You should be able to leave the filenames in place, perhaps with a comment explaining why they are there, centos-release creates files that are missing, but doesn't update them (such that if you set enabled=0 in something like CentOS-Base.repo it should stay that way).
Alternatively, you could change the repos directory. Packages like centos-release will recreate a missing repo file on a system upgrade. However they will not update /etc/yum.conf As such if you control all your repo files you can add:
And place your .repo files in another directory. /etc/yum.repos.d/* will still exist and be updated by system updates, but it will be ignored, only files in /etc/use.these.repos will be used.