{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.1\n",
    "For each type of data, classify it as discrete quantitative, continuous quantitative, categorical, or other.\n",
    "\n",
    "1. How many students are enrolled at a university\n",
    "2. Your favourite day of the week\n",
    "3. How many millimeters of rain fall at an airport during one day\n",
    "4. Weight of a motor vehicle\n",
    "5. Manufacturer of a motor vehicle\n",
    "6. Text of all Google Maps reviews for a restaurant\n",
    "7. Star ratings from all Google Maps reviews for a restaurant\n",
    "8. Size of the living area of an apartment\n",
    "9. DNA nucleotide sequence of a cell\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.2\n",
    "\n",
    "Give the length of each vector or series.\n",
    "\n",
    "1. Morning waking times every day for a week\n",
    "2. Number of siblings (max 12) for each student in a class of 30\n",
    "3. Position and momentum of a roller coaster car"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.3\n",
    "\n",
    "1. Create a NumPy vector called `primes` that consists of the first 8 prime numbers. (You can simply type in the values, nothing fancy needed to compute them.)\n",
    "\n",
    "2. Create a NumPy vector called `squares` that consists of the values [1, 4, 9, 16, ..., 1521, 1600].\n",
    "\n",
    "**Note:** You can use the code in the \"Testing\" cell to verify your solution. If `primes` and `squares` have the correct values, the testing code should run without errors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing \n",
    "assert type(primes) == np.ndarray, \"primes should be a numpy array\"\n",
    "assert primes.shape == (8,), \"primes should be a 1D array with 8 elements\"\n",
    "assert np.sum(primes) == 77, \"Sum of primes should be 77\"\n",
    "assert list(primes) == [2, 3, 5, 7, 11, 13, 17, 19]\n",
    "\n",
    "assert type(squares) == np.ndarray, \"squares should be a numpy array\"\n",
    "assert squares.shape == (40,), \"squares should be a 1D array with 40 elements\"\n",
    "assert np.sum(squares) == 22140, \"Sum of squares should be 22140\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.4\n",
    "\n",
    "1. Create a matrix `C` with 27 rows and 44 columns having all entries equal to 5.2.\n",
    "\n",
    "2. Create a matrix `D` as the 4-by-6 upper-left submatrix of `C`, making sure that `D` is a \"proper\" copy (in the sense that changing an element and one of the matrices does not alter the other matrix)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing\n",
    "assert type(C) == np.ndarray, \"C should be a numpy array\"\n",
    "assert C.shape == (27, 44), \"C should be a 2D array with 27 rows and 44 columns\"\n",
    "assert np.all(np.isclose(C, 5.2)), \"C should have all entries equal to 5.2\"\n",
    "\n",
    "assert type(D) == np.ndarray, \"D should be a numpy array\"\n",
    "assert D.shape == (4, 6), \"D should be a 2D array with 4 rows and 6 columns\"\n",
    "assert np.all(np.isclose(D, 5.2)), \"D should have all entries equal to 5.2\"\n",
    "assert D.flags['OWNDATA'], \"D should be a deep copy, not a view, of C\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.5\n",
    "\n",
    "Create the matrix `R = np.reshape(range(0, 30), (5, 6))`. Use a **single NumPy slice command** to extract the submatrix\n",
    "$$\n",
    "S = \n",
    "\\begin{bmatrix}\n",
    "2 & 3 & 4 \\\\ 14 & 15 & 16 \\\\ 26 & 27 & 28\n",
    "\\end{bmatrix}.\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing\n",
    "assert np.all(S == np.array([[2, 3, 4], [14, 15, 16], [26, 27, 28]])), \"S is incorrect\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.6\n",
    "\n",
    "Write a function `row_col_mean` that accepts a matrix as input and **returns** two vectors: the mean value in every row and the mean value in every column. So, for example, the input\n",
    "$$\n",
    "\\begin{bmatrix}\n",
    "2 & -1 & 2 \\\\ -3 & 1 & 2\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "would produce the outputs $[1, \\: 0]$ and $[-0.5,\\: 0,\\: 2]$. \n",
    "\n",
    "**Important:** When asked to write a function that **returns** values, make sure you do not just use to `print` but instead the `return` keyword. In this exercise, your function should return a tuple with two elements. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing\n",
    "B = np.array([[1, 3, 4],[5, 0, 2]])\n",
    "assert np.all( np.isclose(row_col_mean(B)[0], np.array([2.6666667, 2.3333333])) ), \"Row means are incorrect\"\n",
    "assert np.all( np.isclose(row_col_mean(B)[1], np.array([3, 1.5, 3])) ), \"Column means are incorrect\"\n",
    "testA = np.array([[1,3,-4,-9], [5,0,2,2], [3,5,1,0]])\n",
    "t0, t1 = row_col_mean(testA)\n",
    "assert np.isclose(t0, np.array([-2.25, 2.25, 2.25])).all()\n",
    "assert np.isclose(t1, np.array([ 3., 2.66666667, -0.33333333, -2.33333333])).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.7\n",
    "\n",
    "Write Python code that generates a list `lst = [-1, 1, -1, 1, ..., -1, 1]` with $10^7$ integer elements. Write code that measures the execution time of the command `sum(lst)`, which computes the sum of that list. (There are many ways to measure runtime. You could use the `time()` method in the `time` module. Or check out the `timeit` module.)\n",
    "\n",
    "Now generate a NumPy array from `lst` with the same elements. Then measure and compare the runtime of performing the same summation in NumPy. Can you think of a reason for the difference in performance?\n",
    "\n",
    "Repeat the same experiment but now with a list of floats `[-1.0, 1.0, -1.0, 1.0, ..., -1.0, 1.0]`. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.8\n",
    "\n",
    "Describe a scheme for creating dummy variables for the days of the week. Use your scheme to encode the vector:\n",
    "``` \n",
    "[Tuesday, Sunday, Friday, Tuesday, Monday]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.9\n",
    "\n",
    "Use the following code to load weather data measured at Manchester airport and use calculations on the data frame to assign the correct numerical values to the given variables. You will need to import the appropriate module(s) first.\n",
    "\n",
    "```\n",
    "weather = pd.read_csv(\"_datasets/mcr_airport_weather.csv\")\n",
    "```\n",
    "\n",
    "1. Display the first 7 rows of data frame.\n",
    "\n",
    "2. Verify that the columns `snow` and `tsun` only contain the value `NaN`. Have a look at the CSV file to explain why.\n",
    "\n",
    "3. Use pandas methods to assign values to the following variables:\n",
    "    ```\n",
    "    prcp_june = None   # total precipitation in June (float)\n",
    "    range_sep = None   # difference between maximal and minimal September temp (float)\n",
    "    hottest = None     # hottest day(s) in terms of maximal temperature (dataframe)\n",
    "    ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing\n",
    "assert np.isclose(prcp_june, 48.2), \"prcp_june incorrect\"\n",
    "assert np.isclose(range_sept, 23.0), \"range_sept incorrect\"\n",
    "assert type(hottest) == pd.DataFrame, \"hottest should be a dataframe\"\n",
    "assert hottest.shape[0] == 2, \"hottest should have two rows\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.10\n",
    "\n",
    "Create a frame called `ratings` by loading the file `corporate_rating.csv`. \n",
    "\n",
    "1. Display the first 5 rows. \n",
    "\n",
    "2. The ratings are ordered AAA, AA, A, BBB, BB, B, CCC, CC, C, D. Create a new column called *Rating_number* in which each string value in *Rating* column is replaced with the ordinal equivalents 1, 2, 3, ..., 10.\n",
    "\n",
    "3. How many unique `Name` values (company names) are there?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Testing\n",
    "assert len(ratings.columns) == 32, \"There should be 32 columns\"\n",
    "assert pd.api.types.is_integer_dtype(ratings[\"Rating_number\"]), \"Rating_number should be an integer\"\n",
    "assert ratings[\"Rating_number\"].min() == 1, \"The minimum rating number should be 1\"\n",
    "assert ratings[\"Rating_number\"].max() == 10, \"The maximum rating number should be 10\"\n",
    "assert ratings[\"Rating_number\"].sum() == 8841, \"The sum of rating numbers should be 8841\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2.11\n",
    "\n",
    "There are a number of interesting open data sources in the UK, including \n",
    "* the government's data service at https://www.data.gov.uk/\n",
    "* datasets published by the NHS at https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets\n",
    "\n",
    "In this exercise we will look at some Greater Manchester public transport information from https://www.data.gov.uk/, namely rail station and tram stop Park & Ride spaces. Load the two CSV files `_datasets/rail_park_and_ride_spaces.csv` and `_datasets/Metrolink_Park_and_Ride_Facilities.csv` into pandas data frames `rail` and `metro`, respectively. \n",
    "\n",
    "You will find that each of the data sets has issues. You can use the Jupyter Variable Explorer to explore these. For example, loading the Rail P&R dataset will result in a data frame with Railway Station names that are NaN. The Metrolink dataset, on the other hand, has several of the stop names entered with a question mark `?`, several repetitions of the header line within the data part, and many missing values as well.\n",
    "\n",
    "Unfortunately, such \"messy\" data is not an exception but rather the rule. Hence it is absolutely crucial to spend time investigating the problems and cleaning up the data, before we can draw any conclusions from it. Here are some of the things you could do to improve the situation.\n",
    "\n",
    "**To clean up `rail`:**\n",
    "\n",
    "1. From `rail`, drop all rows where the `Railway Station` is listed as `NaN`\n",
    "\n",
    "2. Set `Railway Station` as the index\n",
    "\n",
    "3. Create a data frame `missing` listing all railway stations that have a missing value in the column `P&R Spaces` \n",
    "\n",
    "4. Create a data frame `rail_clean` listing all railway stations that have a valid value in the column `P&R Spaces` \n",
    "\n",
    "5. Compute the total number of P&R Spaces from `rail_clean` (you will notice that the `P&R Spaces` column needs to be explicitly loaded or converted to numeric)\n",
    "\n",
    "**To clean up `metro`:**\n",
    "\n",
    "1. Remove all question marks `?` in the column `Stop name` \n",
    "\n",
    "2. Set `Stop name` as the index\n",
    "\n",
    "3. Convert the values in `Total parking` to numerical. If this is not possible for a value, then use `NaN`.\n",
    "\n",
    "**Some possible joining operations:**\n",
    "\n",
    "1. Perform an inner join of `rail_clean` and `metro_clean` on their indices to get a single data frame listing the stations/stops that are listed in both data frames.\n",
    "\n",
    "2. Perform an outer join of `rail_clean` and `metro_clean` on their indices to get a single data frame listing the all stations/stops with P&R provision."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}