Currently Apple locks the Siri voice only to the operating system functions. It is a reasonable decision as some app could impersonate Siri and try to steal the sensitive personal data.
However, it is not reasonable that the user cannot unlock Siri voice for other apps if he checks some setting and confirms in the dialog that he is aware of the listed problems that may arise from such an action.
That would bring huge value to many apps. I am the developer of one such app (Speech Central) whose users frequently ask on how to use Siri voice with it. But there are many very popular apps that would have benefit from it, top of the mind it would be Google Maps and nearly all accessibility focused apps.
If you find that this is helpful to you or to some people that you may know, please join me in asking Apple to do this. You can do that using the Apple's feedback assistant, the web link to it is: https://feedbackassistant.apple.com/ . In the header of that page there is the write icon that is used to start the feedback process. Just tell in your own words that you would like the setting to unlock the Siri voice for all apps to people that confirm that they are fine that after that some apps might use the Siri voice into tricking them that it is Siri.
Also as the viewing of the feedback is locked to Apple please also write here a short reply if you think that this idea would be useful.