motivation
Everyone knows that phishing emails are dangerous, and while machine learning is highly effective at detecting them, most modern pipelines act as black boxes. They output a binary decision without explaining why, forcing vulnerable individuals to blindly trust the algorithm.
At the BU RISE program, my partners and I built PhishFence: a detection pipeline that matches the accuracy of leading classifiers while also providing human-readable explanations for its decisions.
Because of the scale of our project, we divided the work, and while my teammates focused on fine-tuning a BERT classifier and integrating LIME and SHAP with Gemini to generate user-friendly explanations, I focused on preparing the foundation. An explainable model is only as robust as the data it is trained on; if the training data is biased, the model will be too. Therefore, my work focused on curating, cleaning, and processing a diverse dataset that made our pipeline reliable.
data preparation
The most well-known online corpus of emails is the Enron corpus, which consists of mostly legitimate emails. We also added several datasets to supplement the number of phishing emails, including the Nazario corpus, for a total of six publicly available datasets.
Although we were able to obtain a CSV file of each dataset, we found that many of the datasets had extraneous columns that were either not shared with the other datasets or were irrelevant to our task. We trimmed each dataset to include only the subject, body, and label columns.
All duplicate rows and rows with missing values were removed. The six trimmed datasets were then concatenated into a single file containing 82,138 entries, of which 39,527 (48%) were legitimate and 42,611 (52%) were phishing. This roughly even split mitigates the risk of biasing the model toward one class or the other.
After initial testing, we determined that the Enron corpus, which comprised 29,767 entries (roughly 36%) of the total dataset, was too large and was skewing the model's performance. In particular, when using LIME to explain model behaviors, the word "Enron" consistently appeared as a top indicator for legitimate emails. This, of course, corresponds with the fact that the Enron corpus as a whole is heavily biased toward legitimate emails containing the word "Enron".
To prevent overfitting to our specific dataset, we replaced all instances of the word "Enron" in the original Enron corpus with a randomly selected company from the Fortune 500 list of companies before recompiling the full dataset. After further analysis, we concluded that this successfully mitigated the bias.
outcome
After training the model using an 80/20 train-test split, we found that our pipeline achieved 99.3% accuracy, with precision and recall both above 99%, matching the performance of leading classifiers. More importantly, our explanability layer was able to provide human-readable explanations for each classification decision.
We presented our results at the RISE Poster Symposium, where we showcased a web app where a user pastes in an email for classification and explanation.
limitations
When building a dataset for phishing, one major consideration is the ratio of legitimate to phishing emails. Class imbalance can lead to heavily skewed models, but artificially balancing the dataset can also lead to prediction bias. In our case, we chose to use a larger set of phishing emails to better capture the diverse nature of phishing attacks. As a result, we must understand that the resulting model may be biased toward predicting phishing emails.
Additionally, our final pipeline makes use of Google's BERT language model, which can only read 512 tokens at a time. This means that we had to chunk longer emails, causing degraded context understanding and potentially decreased accuracy for longer emails.
Our dataset also present inherent limitations: all the emails are in English, and the training corpus is static. This means that the model may not generalize well to other languages or to new phishing techniques that emerge after the dataset was compiled. As a proof-of-concept, PhishFence is a strong starting point, but it is not a production-ready system.
future work
Future research could focus on expanding the training data to include more languages and more recent phishing attacks. Additionally, the model could be improved to handle longer emails without chunking, perhaps by using a different architecture or by fine-tuning BERT for longer sequences. Finally, user studies could be conducted to evaluate the effectiveness of PhishFence's explanations in helping users make informed decisions about potential phishing emails.