Shapley value (phi) and feature importance provide different views on the model.

Shapley value illustrates the ‘payout’ of the feature within the model and the direction of its impact on the classification of pneumonia. The difference between the actual and average prediction implies the sum of the payout for prediction is 0.10. Benzodiazephine has a larger payout that alteplase (RTPA) or beta-blockers on pneumonia. Feature importance illustrates the impact on the model’s error by permuting the features. BNIH = baseline NIHSS, BENZ2 = benzodiazepine, BB2 = beta blockers, SLEE2 = non-benzodiazepine sleeping medications, IHD = ischemic heart disease, CHF, congestive heart failure, AFIB = atrial fibrillation, RTPA = recombinant tissue plasminogen activator.