Extracting Feature Importances from a Classifier Model

How feature importances aid your classification projects

Image for post
Image for post
Photo by Joshua Sortino on Unsplash

Extracting Feature Importances

# Instantiate the classifier model:
rf_clf = RandomForestClassifier()
# Fit the model to the training data:
rf_clf.fit(X_train, y_train)
# Extract the Feature importances from the model:
importance = model.feature_importances_
# Linear Regression:
importance = model.coef_
# Logistic Regression:
importance = model.coef_[0]

Arranging the Values in a readable output

# Output importances 
imp_list = []
for i,v in enumerate(importance):
imp_list.append((i, v))
sorted_list = sorted(imp_list, key=lambda x: x[1], reverse=True)
print('The top 10 Feature importances are:' '\n')for f, i in sorted_list[:10]:
print(f'{X_train.columns[f]}, Score: {round(i, 3)}')
# Plot importances:
plt.bar([x for x in range(len(importance))], importance)
plt.ylim(0, 0.08)
plt.show()
Image for post
Image for post
All feature importances
# Plot top ten importances
plt.bar(range(1, 11), [x[1] for x in sorted_list[:10]],
tick_label=[x[0] for x in sorted_list[:10]],
color = (0.2,0.5,0.7,0.6))
plt.title('Top Ten Feature Importances')
plt.xlabel('Feature Number')
plt.ylabel('Feature Importance Value')
Image for post
Image for post
Top ten feature importances

How do Feature Importances aid Feature Selection

from sklearn.feature_selection import SelectFromModel

Conclusion

References

Practicing Data Scientist. Interested in Games, Gamification, Ocean Sciences, Music, Biology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store