Chicago Crime - Updated

I was a bit dismayed to discover that the EveryBlock API stopped working. One of my most popular posts, Chicago Crime, depended on this API heavily.

Here I present an alternative data source for my previous post and a fun update. The update is a quick and dirty method to create popular crime density plots, as shown below.

In the spirit of transparency, the city of Chicago has taken the initiative to electronically release a great deal of data with Socrata Open Data API (SODA). Luckily for us, this release includes all filed police reports. The records available through this API range from the year 2001 to the present, which dwarfs the two week range provided by EveryBlock's API. However, I would not recommend downloading all of this data as the text file is approximately 2GB in size.

In this post I use only the filed police reports from one year prior to present. The data is available in both JSON and CSV format. I only use the CSV format here for the convenience of those with an earlier version of Mathematica. To import this data into Mathematica:

A plot of all reported crimes in Chicago between February 1st and March 31st of 2012 was created with the data.

Plot of incidents for any period within the year can be easily created with the following Mathematica code. Restrict the data to the range of interest by replacing the dates of the start and end strings. Dates must be in the format of mm/dd/yyyy. This code simply finds the positions of the strings that match the inputed dates and plots the daily totals of incidents.

The data can be parsed into primary descriptions including arson, homicide, sex offense, weapons violation, crim sexual assault, interfere with public officer, gambling, public peace violation, criminal trespass, liquor law violation, assault, deceptive practice, burglary, criminal damage, robbery, motor vehicle theft, offense involving children, narcotics, theft, other offense, and battery. Below is the plot and code to filter for only narcotics incidents.

A secondary description can also be used to parse the data. Some secondary descriptions include aggravated: handgun, poss: crack, armed: handgun, harassment by telephone, poss: heroin(white), to land, retail theft, unlawful entry, strongarm - no weapon, to property, poss: cannabis 30gms or less, from building, $500 and under, forcible entry, to vehicle, domestic battery simple, simple, automobile, over $500, telephone threat, and wireroom/sports. Below is the plot and code to filter for incidents with a primary description of narcotics and a secondary description of poss: cannabis 30gms or less.

The incidents may also be separated into ones where arrests were made and ones where no arrests were made. Below is code to filter for incidents with a primary description of narcotics and a secondary description of poss: cannabis 30gms or less with arrests and without. Both plots comparing to total narcotics incidents are shown.

The method to display where certain incidents occur in chicago is similar to my previous post, Chicago Crime. Please refer to that posting for a more detailed explanation on how to create the plot below. The plot here is a geographical visualization of where arrests occurred relating to incidents of narcotics with possession of cannabis with 30 grams or less.

The method to tabulate the number of incidents within neighborhoods is also similar to my previous post, Chicago Crime. This calculation totals the numbers of incidents within each neighborhood boundary. It is the most costly in computational time and power because of the generality in the counting code that allows it to use any boundaries (such as census tracts, police beats, etc.) The tabulation of arrests made relating to incidents of narcotics with possession of cannabis with 30 grams or less in each neighborhood is presented.

While doing some initial research for this post, I found "crime hotspots" as a popular visualization option. A similar visualization is easily created in mathematica with three simple lines of code.

The Mathematica code used in this post can be downloaded here. Some fun analysis of crimes in chicago will follow in future posts.

Making Sense of Census-less Data

I have always found that supplemental 3D visualizations help provide better insights on census data. While playing with the compiled data from the 2000 census, from cityofchicago.org, I found an interesting anomaly.

After some investigation, I found the red spike represents 15 families with a median income of $255,000. Was there mischief afoot by those 15 families?

I'll let the more curious readers decide. Attached here is the census data and a Mathematica notebook. The notebook has been generalized to allow users to create 3D visualizations of all provided census data.

Download here

Chicago Crime

As a former resident of Chicago I was curious to how much crime happened in the period between  Feb. 18, 2011 and May 15, 2011 and where crimes occurred.  Of course this analysis could be done on EveryBlock, but that is just too easy.  I wanted to do it in Mathematica, incase I want to do any type of advanced analysis later.

The GIS information for the city of Chicago can be found here.  The data must then be unzipped and converted to an XML format, which can be done with shp2text.  The data that is necessary to for this analysis is provided in the downloadable zip file found below.

The first step to the analysis is to extract the GIS data of the city.  To extract the x and y coordinates that make up the polygons of the city of Chicago, the following command was used:

The data used here were of Chicago Police Distracts and historic neighborhoods of Chicago.

The next step was to access the EveryBlock API and gather the crime data.  The API is free to access, you only need to register for a key to use it.  A quirky characteristic about this API is the limited data that is provided.  In the request command I have requested all crimes from January 1st, 2010 but the API will only provide crimes uploaded from the last 24 hours from EveryBlock.  The command to access and isolate the crime data is:

The odd thing about this project is the coordinate disparity.  The Chicago GIS data is provided in a NAD83 transformation rather than just latitudes and longitudes.  This transformation gives x and y coordinates in units of feet rather than degrees and minutes.  To convert the latitude and longitude data gathered from the EveryBlock API this Mathematica command was used:

Now let's get to some pretty pictures and results.  For the period between Feb. 18th, 2011 to May 15th, 2011, the crime distribution in Chicago looked like this:

A count of crime incidents can be performed since we are doing this in Mathematica.  The number of crime incidents in each neighborhood are listed in the table below:

The same analysis can be done with Chicago Police Districts.  CPD publishes monthly crime statistics for each district, I have compiled this information for the month of January 2011 to add a bit more color to the crime map.  In the crime map below, more red the color the higher the incidents of crime in the district.

And the accompanying crime incidents for each district is:

The datafiles used in this blog is provided here.