https://machinelearningmastery.com/custom-metrics-deep-learning-keras-python/
https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/
0. Classification Label
- One-hot Label (Default): [0,0,1] [0,1,0] [1,0,0]
- Sparse Label: [1] [2] [3]
1. Losses
1.1 Regression
- Mean Squared Error Loss: mse, or mean_squared_error
- Mean Squared Logarithmic Error Loss: mean_squared_logarithmic_error, or msle
- Mean Absolute Error Loss: mean_absolute_error, or mae
- Others: mape
1.2 Binary Classification
- Binary Cross-Entropy: binary_crossentropy
- Hinge Loss: hinge
- Squared Hinge Loss: squared_hinge
1.3 Multi-Class Classification
- Multi-Class Cross-Entropy Loss: categorical_crossentropy
- Sparse Multiclass Cross-Entropy Loss: sparse_categorical_crossentropy
- Kullback Leibler Divergence Loss: kullback_leibler_divergence
2 Metrics
2.1 Regression Metrics
- Mean Squared Error: mean_squared_error, MSE or mse
- Mean Absolute Error: mean_absolute_error, MAE, mae
- Mean Absolute Percentage Error: mean_absolute_percentage_error,
MAPE, mape - Cosine Proximity: cosine_proximity, cosine
2.2 Binary Classification Metrics
- Binary Accuracy: binary_accuracy, acc
- Sparse Categorical Accuracy: sparse_categorical_accuracy
2.3 Multi-Classification Metrics
- Categorical Accuracy: categorical_accuracy, acc
- Sparse Categorical Accuracy: sparse_categorical_accuracy
- Top k Categorical Accuracy: top_k_categorical_accuracy (requires you specify a k parameter)
- Sparse Top k Categorical Accuracy: sparse_top_k_categorical_accuracy (requires you specify a k parameter)