Global Terrorism Analysis

Predicting the Success of Terrorist Attacks

Terrorism is a worldwide problem. Between 1970 and 2017, there were 181,691 terrorist attacks recorded globally. Attacks have been recorded on every continent and in over 180 countries. In the interest of national security, we will analyze terrorist attacks between 1997 and 2017 to figure out the specific factors that determine whether a terrorist attack will be successful. By knowing the patterns, strengths and weakness of terrorists and terrorist organizations, we can be more prepared to prevent new attacks in the future.

Throughout the course of this notebook, we will attempt to determine what factors are most important in ensuring the success of a terrorist attack. Utilizing the OSEMN data science process, we will analyze the data and then employ various machine learning algorithms to determine the importance of features.

Defining “Success” According to the Global Terrorist Database:
Success of a terrorist strike is defined according to the tangible effects of the attack. Success is not judged in terms of the larger goals of the perpetrators. For example, a bomb that exploded in a building would be counted as a success even if it did not succeed in bringing the building down or inducing government repression.

The definition of a successful attack depends on the type of attack. Essentially, the key question is whether or not the attack type took place. If a case has multiple attack types, it is successful if any of the attack types are successful, with the exception of assassinations, which are only successful if the intended target is killed.

Data

The data used in this analysis is from the Global Terrorism Database. This database was compiled by the National Consortium for the Study of Terrorism and Responses to Terrorism. Aside from coming up with a catchy name for their organization, they’ve also compiled data on over 180,000 terrorist attacks worldwide between 1997 and 2017. The full dataset contains 181,691 rows and 135 columns.

Scrubbing

Upon initial inspection, there were several methods we were able to use to make the dataset more manageable. First, we decided to focus on just the most recent 20 years of the dataset (1997–2017) and this brought the number of rows down to just over 117,000. Several columns were irrelevant to the target, several others were redundant, and many were able to be condensed into a single feature. The final pruned dataset was 117,381 rows and 42 columns.

From here, we split the data into training and testing sets using train_test_split.

X = df.drop('success', axis = 1)
y = df['success']
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 30)

Next, we separated the training data into numerical and categorical columns as well as created Pipelines to transform the data for modeling.

num_transformer = Pipeline(steps = [('imputer', KNNImputer(n_neighbors = 2)), ('scaler', StandardScaler())])
cat_transformer = Pipeline(steps = [
    ('imputer', SimpleImputer(strategy = 'most_frequent')), 
    ('encoder', OneHotEncoder(handle_unknown = 'ignore', sparse = False))])

preprocessing = ColumnTransformer(transformers = [
    ('num', num_transformer, num_cols),
    ('cat', cat_transformer, cat_cols)])

X_train_trans = preprocessing.fit_transform(X_train)
X_test_trans = preprocessing.transform(X_test)

Explore

We employed a variety of plots to find interesting trends within the data.

Map of Attempted Terrorist Attacks

Above is a map indicating the location of every terrorist attack from 1970 to 2017.

Geographic Region

The plot above shows that an overwhelming majority of terrorist attacks occur in the Middle East/North Africa and South Asia.

Attack Type

The plot above shows that an most attempted terrorist attacks are either bombings or armed assaults.

Target Type

The plot above shows that the top targets for a terrorist attack are Private Property, Military, Police, Government, and Business.

Modeling

The modeling phase was an iterative process where we ran several different types of models in order to determine the best fit for the data.

Logistic Regression

First on the list is Logistic Regression. First, we run a base model and then performed a GridSearchCV in order to tune the hyperparameters. After getting the optimal hyperparameters, we fit a seocnd model and reviewed our results.

params = {'class_weight': ['balanced'],
          'solver': ['lbfgs', 'liblinear'],
          'C': [1.0, 3.0, 5.0]}
grid = GridSearchCV(estimator = LogisticRegression(), param_grid = params, cv = 3, n_jobs = -1)
grid.fit(X_train_trans_df, y_train)

logreg2 = LogisticRegression(class_weight = 'balanced',
                             C = 1.0,
                             solver = 'lbfgs')
logreg2.fit(X_train_trans_df, y_train)

Decision Tree

Next, we ran a base decision tree classifier to see if this would return better results. Again, we then performed a GridSearchCV to find the optimal hyperparameters and then fit a second model.

params = {'class_weight': [None, 'balanced'],
          'criterion': ['gini', 'entropy'],
          'max_depth': [1, 3, 5], 
          'min_samples_leaf': [1, 3, 5]}
grid = GridSearchCV(estimator = DecisionTreeClassifier(), param_grid = params, cv = 3, n_jobs = -1)
grid.fit(X_train_trans_df, y_train)

decision_tree2 = DecisionTreeClassifier(class_weight = 'balanced', 
                                        criterion = 'entropy', 
                                        max_depth = 5, 
                                        min_samples_leaf = 5)
decision_tree2.fit(X_train_trans_df, y_train)

Random Forest

After the decision tree, we decided to try an ensemble method and used the random forest algorithm. Once again, we performed a GridSearchCV and fit the model with the tuned hyperparameters.

params = {'class_weight': [None, 'balanced'],
          'criterion': ['gini', 'entropy'],
          'max_depth': [1, 3, 5], 
          'min_samples_leaf': [1, 5, 10]}
grid = GridSearchCV(estimator = RandomForestClassifier(), param_grid = params, cv = 3, n_jobs = -1)
grid.fit(X_train_trans_df, y_train)

random_forest2 = RandomForestClassifier(class_weight = 'balanced', 
                                        criterion = 'entropy', 
                                        max_depth = 5, 
                                        min_samples_leaf = 10)
random_forest2.fit(X_train_trans_df, y_train)

XGBoost

Next, we wanted to experiment with using XGBoost on the data. In case you’re not noticing the pattern yet, we performed a GridSearchCV and fit the model with the tuned hyperparameters.

Stacking Ensemble

The results are very good, but we wanted to see if we would be able to get a bit more accuracy by using a Stacking Ensemble made up of our decision tree, random forest, and XGBoost.

estimators = [('dt', DecisionTreeClassifier(class_weight = 'balanced', 
                                            criterion = 'entropy',
                                            max_depth = 5, 
                                            min_samples_leaf = 5)),
              ('rf', RandomForestClassifier(class_weight = 'balanced', 
                                            criterion = 'entropy', 
                                            max_depth = 5, 
                                            min_samples_leaf = 10)), 
              ('xg', xgb.XGBClassifier(gamma = 1,
                                       min_child_weight = 1,  
                                       max_depth = 5))]

stack = StackingClassifier(estimators = estimators, cv = 3, n_jobs = -1)
stack.fit(X_train_trans_df, y_train)

Interpret

In terms of trying to prevent terrorist attacks, it is extremely important that we limit the false negatives in our model. Therefore, the metric that we chose to use for scoring the model performence is the Recall.

Though all the models performed well, we can see that XGBoost and the Stacking Classifier performed the best in terms of Recall Score. Though the Stacking Classifier performed better in other ways, we will be moving forward with the XGBoost model. The reason for this is that we will be using SHAP for further analysis and at this moment, SHAP does not support Stacking Classifiers.

Speaking of SHAP…here’s a SHAP bar plot showing the most important features as per SHAP calculations.

…and here is a SHAP plot showing the positive and negative effect on the target per each important feature.

…and now a simple bar plot showing the most important features based on the outputs from the model.

Results

Top 5 Feature Importances from Modeling — targtype1_txt_Unknown, ishostkid, property, nkill, attacktype1_txt_Assassination

Top 5 Feature Importances from SHAP — nkill, property, nwound, attacktype1_txt_Assassination, ishostkid

Overlapping Important Features — nkill, ishostkid, attacktype1_txt_Assassination, property

Overlapping Important Features Detailed:
1. nkill — Total Number of Fatalities — High numbers of fatalities positively affect the success and low numbers of fatalities negatively affect the success of attack.

2. ishostkid — Hostages or Kidnapping Victims — Very positively affects the success of attack.

3. attacktype1_txt_Assassination — Attempted Assassination — Very negatively affects the success of attack.

4. property — Property Damage (Evidence of property damage from attack) — Slightly positively affects the success of attack.

Recommendations

Based on the results of this analysis, we can make the following recommendations:

• Maintain high levels of intel and security in the Middle East and South Asia.
• Develop better methods and/or technology for bomb detection and disarmament.
• Focus intel on target areas that have the highest concentration of people.
• Increase security for high level targets for potential kidnapping or hostage situations.

Future Work

With more time, we can gain even more insight into what can make a terrorist attack successful in order to create better security measures.

• Time: We can increase the range of years of the data in our analysis. For example, the full dataset ranges from 1970 to 2017.
• Models: We can increase the size and complexiy of our models in order to increase the accuracy of our results.
• Data: We can research and compile additonal data from other resources for a more well rounded dataset.