The problem:
We recently enabled peer cache in a large environment running SCCM/MEMCM 1906. A package that was previously tested to install successfully now started failing to install. Clients would reach downloading 99% before failing out without ever running the installation. CAS.log showed hash verification failures, and ContentTransferManager.log always showed that the content was downloaded from a peer source. Anti-Virus exclusions, etc. were proven out to be in place and working correctly, and so we added a bogus file to the package in order to get DistMgr to resnapshot the package and redistributed the content out. We then pushed out machine policy and deployment evals, and clients started successfully installing the application. This seemed to be resolved, however, the next day we were right back where we were before with hash validation issues.

The smoking gun:
The application's installer was outputting a setup.log into its running directory, causing the hash of the package to be different than it was prior to installing. When peers would attempt to use the super peer that had already run the install as a content source, they were downloading a package with a different hash than the content library had for the package causing it to fail hash validation.

Possible fixes:

1. Pass the setup executable a path to output its setup log into a different location than CCMCache
2. In the scripted package install, add code to first copy the installer to a temporary location. Change the script to call the installer executable from the temporary location instead of from CCMCache, then remove the temporary copy of the content.

In this case, the setup executable wasn't honoring a switch to output its setup log elsewhere, but option 2 cleared it up.

Summary:
In an environment without peer cache, this issue wouldn't really be noticed, because once the install is complete, the content isn't ever used by ConfigMgr/SCCM/MEMCM again. With peer cache in play, however, we need to keep focus on preserving the content in its original form through and after an installation because other clients must download an exact copy in order to use the content themselves.