Add lesson 7 - activity 1

Author: netmario

lesson07/Activity01/activity01.ipynb

@@ -0,0 +1,263 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# New York City\n",
+    "In this activity we will visualize data about New York City (NYC) and compare it with the state of New York and the United States (US). The American Community Survey (ACS) Public Use Microdata Sample (PUMS) dataset (1-year estimate from 2017) from https://www.census.gov/programs-surveys/acs/technical-documentation/pums/documentation.2017.html is used.\n",
+    "\n",
+    "Download the following datasets and place the extracted csv-file in the data subdirectory:\n",
+    "https://www2.census.gov/programs-surveys/acs/data/pums/2017/1-Year/csv_pny.zip\n",
+    "https://www2.census.gov/programs-surveys/acs/data/pums/2017/1-Year/csv_hny.zip\n",
+    "\n",
+    "In this activity the datasets 'New York Population Records' (./data/pny.csv) and 'New York Housing Unit Records' (./data/hny.csv) are used. The first dataset contains information about the New York population, and the second dataset contains information about housing units. The dataset contains data for about 1% of the population and housing units. Due to the extensive amount of data we do not provide the datasets for the whole US, instead we provide the required information related to the US if necessary. The pdf 'PUMS_Data_Dictionary_2017.pdf' gives an overview and description of all variables. A further description of the codes can be found in 'ACSPUMS2017CodeLists.xls'."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Use the following code cell to define all required import statements."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Import statements\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Use pandas to read both csv-files located in the subdirectory 'data'."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Use the given PUMA (public use microdata area code based on 2010 Census definition which are areas with populations of 100,000 or more) ranges to further divide the dataset into NYC districts (Bronx, Manhatten, Staten Island, Brooklyn, and Queens)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# PUMA ranges\n",
+    "bronx = [3701, 3710]\n",
+    "manhatten = [3801, 3810]\n",
+    "staten_island = [3901, 3903]\n",
+    "brooklyn = [4001, 4017]\n",
+    "queens = [4101, 4114]\n",
+    "nyc = [bronx[0], queens[1]]\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "In the dataset each sample has a certain weight that reflects the weight for the total dataset. Therefore we cannot simply calculate the median. Use the given weighted_median function in the following to compute the median."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Function for a 'weighted' median\n",
+    "def weighted_frequency(values, weights):\n",
+    "    weighted_values = []\n",
+    "    for value, weight in zip(values, weights):\n",
+    "        weighted_values.extend(np.repeat(value, weight))\n",
+    "    return weighted_values\n",
+    "\n",
+    "def weighted_median(values, weights):\n",
+    "    return np.median(weighted_frequency(values, weights))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Wages\n",
+    "In this subtask we will create a plot containing multiple subplots which visualize information with regards to NYC wages.\n",
+    "- Visualize the median household income for the US, New York, New York City, and its districts.\n",
+    "- Visualize the average wage by gender for the given occupation categories for the population of NYC.\n",
+    "- Visualize the wage distribution for New York and NYC. Use the following yearly wage intervals: 10k steps between 0 and 100k, 50k steps between 100k and 200k, and >200k"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data wrangling for median housing income\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data wrangling for wage by gender for different occupation categories\n",
+    "occ_categories = ['Management,\\nBusiness,\\nScience,\\nand Arts\\nOccupations', 'Service\\nOccupations',\n",
+    "                 'Sales and\\nOffice\\nOccupations', 'Natural Resources,\\nConstruction,\\nand Maintenance\\nOccupations',\n",
+    "                 'Production,\\nTransportation,\\nand Material Moving\\nOccupations']\n",
+    "occ_ranges = {'Management, Business, Science, and Arts Occupations': [10, 3540], 'Service Occupations': [3600, 4650], \n",
+    "                 'Sales and Office Occupations': [4700, 5940], 'Natural Resources, Construction, and Maintenance Occupations': [6000, 7630], \n",
+    "                 'Production, Transportation, and Material Moving Occupations': [7700, 9750]}\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data wrangling for wage distribution\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Create figure with four subplots\n",
+    "\n",
+    "\n",
+    "# Median household income in the US\n",
+    "us_income_median = 60336\n",
+    "\n",
+    "# Median household income\n",
+    "\n",
+    "\n",
+    "# Wage by gender in common jobs\n",
+    "\n",
+    "\n",
+    "# Wage distribution\n",
+    "\n",
+    "\n",
+    "# Overall figure\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Occupations\n",
+    "Use a tree map to visualize the percentage for the given occupation subcategories for the population of NYC."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data wrangling for occupations\n",
+    "occ_subcategories = {'Management,\\nBusiness,\\nand Financial': [10, 950],\n",
+    "                    'Computer, Engineering,\\nand Science': [1000, 1965],\n",
+    "                    'Education,\\nLegal,\\nCommunity Service,\\nArts,\\nand Media': [2000, 2960],\n",
+    "                    'Healthcare\\nPractitioners\\nand\\nTechnical': [3000, 3540],\n",
+    "                    'Service': [3600, 4650],\n",
+    "                    'Sales\\nand Related': [4700, 4965],\n",
+    "                    'Office\\nand Administrative\\nSupport': [5000, 5940],\n",
+    "                    '': [6000, 6130],\n",
+    "                    'Construction\\nand Extraction': [6200, 6940],\n",
+    "                    'Installation,\\nMaintenance,\\nand Repair': [7000, 7630],\n",
+    "                    'Production': [7700, 8965],\n",
+    "                    'Transportation\\nand Material\\nMoving': [9000, 9750]}\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Visualization of tree map\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Correlation\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Use a heatmap to show the correlation between difficulties (self-care difficulty, hearing difficulty, vision, difficulty, independent living difficulty, ambulatory difficulty, veteran service connected disability, and cognitive difficulty) and age groups (<5, 5-11, 12-14, 15-17, 18-24, 25-34, 35-44, 45-54, 55-64, 65-74, 75+) in New York City."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Data wrangling for New York City population difficulties\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Heatmap\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.6"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}