The appraiser did the work and so the appraiser must be paid. The work was done on behalf of the borrower and so it makes sense that the borrower should pay for the appraisal even if the transaction does not close.
Prudent appraisers are asking for payment at the door before they even start the appraisal.
It's the appraiser's job to place a value on the property and it's entirely possible that the appraisal report will cause the deal to fall apart. The appraiser must still be paid.