Introduction

The Boston Crime Incidents Report is a dataset released by the city of Boston via the Boston open data portal which contains information about crimes reported to the BPD over the last few years (4/30/2011 - 6/03/2014).

The version of the dataset that I downloaded consists of 320,574 crimes reported during this period, and includes information about the nature of the incident (“Homicide”, “Larceny”, “Assault”, etc.), date & time of the report, as well as the location.

In this post, we’ll take a look at the geospatial aspect of the data, looking at how crime reports are geographically distributed in Boston.

Cleaning Up the Data

To start, let’s take a closer look at the data set. Here is a look at some of the columns in the data set and the first few records in the table:

> crime <- read.csv("Crime_Incident_Reports.csv")
> colnames(crime)
> [1] "COMPNOS" "NatureCode"  
>  [3] "INCIDENT_TYPE_DESCRIPTION" "MAIN_CRIMECODE"  
>  [5] "REPTDISTRICT" "REPORTINGAREA"  
>  [7] "FROMDATE" "WEAPONTYPE"  
>  [9] "Shooting" "DOMESTIC"  
> [11] "SHIFT" "Year"  
> [13] "Month" "DAY_WEEK"  
> [15] "UCRPART" "X"  
> [17] "Y" "STREETNAME"  
> [19] "XSTREETNAME" "Location"
> head(crime)

    COMPNOS NatureCode INCIDENT_TYPE_DESCRIPTION MAIN_CRIMECODE

1 110220085 MVANO MVAcc MVAcc
2 110219791 CARST DEATH INVESTIGATION 01INV
3 110219155 UNCONS MedAssist MedAssist
4 110219159 BEIP BENoProp BENoProp
5 110219167 NIDV OTHER LARCENY 06xx
6 110219169 PERGUN Argue Argue
REPTDISTRICT REPORTINGAREA FROMDATE WEAPONTYPE Shooting
1 B2 320 04/30/2011 05:54:00 AM Unarmed No
2 A7 21 04/30/2011 06:00:00 AM Other No
3 E5 698 04/30/2011 06:21:00 AM Unarmed No
4 D4 620 04/30/2011 06:24:00 AM Unarmed No
5 B2 181 04/30/2011 06:50:00 AM Unarmed No
6 C11 351 04/30/2011 06:53:00 AM Firearm No
DOMESTIC SHIFT Year Month DAY_WEEK UCRPART X Y
1 No Last 2011 4 Saturday Part Three 768922.2 2938691
2 No Last 2011 4 Saturday Part Three 781735.0 2961811
3 No Last 2011 4 Saturday Part Three 751855.0 2928001
4 No Last 2011 4 Saturday Part Two 767168.1 2951333
5 Yes Last 2011 4 Saturday Part One 771778.9 2944026
6 No Last 2011 4 Saturday Part Three 775421.2 2934394
STREETNAME XSTREETNAME Location
1 HOWLAND ST WARREN ST (42.31116136, -71.08312956)
2 PARIS ST (42.37442134, -71.03529458)
3 W ROXBURY PY (42.28203873, -71.14639217)
4 HEMENWAY ST (42.34587634, -71.08938955)
5 GEORGE ST (42.32576135, -71.07246956)
6 SALISBURY PARK (42.29928135, -71.05918456)

We see that the data contains one record for each incident, as well as a number of other structured fields regarding the incident.

To plot the data on a map, we will make use of the “Location” field. However, we first need to convert this field into a more useful form–in particular, we need to split the latitude and longitude into two separate columns. To do this, we can use the stringr package with a bit of regex like so:

require(stringr)

> regex <-"\\((\\d+\\.\\d+), (-\\d+\\.\\d+)\\)"
> lat_long <- str_match(crime$Location, regex)[,2:3]
> lat_long <- apply(lat_long, 2, as.numeric)
> colnames(lat_long) <- c("y", "x")
> crime_latlong <- cbind(crime, lat_long)

> head(lat_long) #much better

            y         x

[1,] 42.31116 -71.08313
[2,] 42.37442 -71.03529
[3,] 42.28204 -71.14639
[4,] 42.34588 -71.08939
[5,] 42.32576 -71.07247
[6,] 42.29928 -71.05918

Exploring the Data with ggmap

With the location field cleaned up, we can now get to plotting. For this, we will use the ggmap package, which uses a ggplot-like grammar for easily plotting geospatial data. As a first exercise, here is the geographical distribution of drug charges in Boston:

# extract instances of drug charges

drug_crimes <- crime_latlong[crime_latlong$INCIDENT_TYPE_DESCRIPTION == "DRUG CHARGES",]

# plot on a map

require(ggmap)
map.center <- geocode("Boston, MA")
Bos_map <- qmap(c(lon=map.center$lon, lat=map.center$lat), zoom=12)
g <- Bos_map + geom_point(aes(x=x, y=y), data=drug_crimes, size=3, alpha=0.2, color="red") +
ggtitle("Drug Charges in Boston by Location (2011-2014)")

center

If you’re famililar with ggplot, you’ll note that ggmap is conceptually very similar. In this case Bos_map is my base layer map of Boston (centered on the lat-long determined by geocode). I then use geom_point to add the crime data layer to the map. If you want to learn more, a good introduction to ggmap can be found here.

As for the plot itself, a couple things are of note. For one thing, you may notice that a few regions of the city are conspicuously lacking in crime. In particular, there appear to have been exactly zero drug charges in Cambridge and Somerville over the last few years; likewise for Brookline. Of course this isn’t correct, but is rather a result of the fact that these areas are not part of the City of Boston proper, and hence were not included in the dataset (i.e. they have their own police departments).

To clarify this point, I thought it might make sense to visualize the exact geographic extent of the City of Boston. To do this, I pulled down the Boston Neighborhood Shapefiles from the open data portal. I have no previous experience working with shapefiles, but after a bit of googling (e.g.) and experimentation (and installing some new packages), I was able to plot them with ggmap:

#load R geo packages
require(rgdal)
require(sp)

#read the shape files
datadir <- "./Bos_neighborhoods_new/"
neighbs <- readOGR(dsn=datadir, layer="Bos_neighborhoods_new")

#prepare for plotting
neighbs <- spTransform(neighbs, CRS("+proj=longlat +datum=WGS84"))
neighbs_plt <- fortify(neighbs)

#plot the neighborhoods with ggmap
Bos_map2 <- qmap(c(lon=map.center$lon, lat=map.center$lat), zoom=11)
Bos_map2 + geom_polygon(data=neighbs_plt, aes(x=long, y=lat, group=group), alpha=0.3, color="black", fill='red') + ggtitle("Geographic Extent of Boston")

center

Each of the red-shaded polygons in this image represent a different neighborhood in the City of Boston. If we overlay this map with our previous drug charges plot, we see that, as expected, our dataset is entirely contained in this area (drug charges now represented in black):

# plot neighborhoods and crimes

Bos_map2 + geom_polygon(data=neighbs_plt, aes(x=long, y=lat, group=group), alpha=0.3, color="black", fill='red') +geom_point(aes(x=x, y=y), data=drug_crimes, size=2, alpha=0.2, color="black")+
ggtitle("Geographic Extent of Boston with Drug Charges (2011-2014) Overlay")

center