refactor: migrate exploratory data analysis examples to polars

.gitignore

@@ -4,7 +4,7 @@
 # quarto
 _book/
 /.quarto/
-**/*.quarto_ipynb
+**/*.quarto_ipynb*
 _freeze/
 .panache*
 
@@ -137,3 +137,5 @@ dmypy.json
 
 # Pyre type checker
 .pyre/
+
+**/*.quarto_ipynb

boolean-data.ipynb

@@ -11,9 +11,13 @@
     "\n",
     "In this chapter, we'll introduce boolean data: data that can be `True` or `False` (which can also be encoded as 1s or 0s). We'll first look at the fundamental Python true and false boolean variables before seeing how true and false work in data frames.\n",
     "\n",
+    "### Prerequisites\n",
+    "\n",
+    "This chapter focuses on working with boolean data and expressions using **polars**. Make sure you have **polars** installed (`uv add polars`). For more on Polars boolean operations, see the [Polars boolean expression documentation](https://docs.pola.rs/api/python/stable/reference/expressions/boolean.html).\n",
+    "\n",
     "## Booleans\n",
     "\n",
-    "Some of the most important operations you will perform are with `True` and `False` values, also known as boolean data types. These are fundamental Python variables, just as numbers such as `1` are. \n",
+    "Some of the most important operations you will perform are with `True` and `False` values, also known as boolean data types. These are fundamental Python variables, just as numbers such as `1` are.\n",
     "\n",
     "### Boolean Variables and Conditions\n",
     "\n",
@@ -481,11 +485,11 @@
    "id": "8e06930b",
    "metadata": {},
    "source": [
-    "## Booleans in **pandas** data frames\n",
+    "## Booleans in **polars** DataFrames\n",
     "\n",
     "### Operations on booleans in data frames\n",
     "\n",
-    "Quite often, you will run into a scenario where you're working with data that have True or False values in a data frame. It is easy to create a column of booleans in a **pandas** data frame:"
+    "Quite often, you will run into a scenario where you're working with data that have True or False values in a data frame. It is easy to create a column of booleans in a **polars** data frame:"
    ]
   },
   {
@@ -495,9 +499,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import pandas as pd\n",
+    "import polars as pl\n",
     "\n",
-    "df = pd.DataFrame.from_dict(\n",
+    "df = pl.DataFrame(\n",
     "    {\n",
     "        \"bool_col_1\": [False] * 3 + [True, True],\n",
     "        \"bool_col_2\": [True, False, True, False, True],\n",
@@ -511,7 +515,7 @@
    "id": "c7a0cd32",
    "metadata": {},
    "source": [
-    "We can perform operations on these just like regular **pandas** data frame columns. These accept `&` (and), `|` (or), `==` (equal), and `!=` (not equal) as operations:"
+    "We can perform operations on these just like regular **polars** data frame columns. These accept `&` (and), `|` (or), `==` (equal), and `!=` (not equal) as operations:"
    ]
   },
   {
@@ -521,15 +525,20 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df[\"bool_col_1\"] | df[\"bool_col_2\"]"
+    "df.select(\n",
+    "    [\n",
+    "        (pl.col(\"bool_col_1\") & pl.col(\"bool_col_2\")).alias(\"and_result\"),\n",
+    "        (pl.col(\"bool_col_1\") | pl.col(\"bool_col_2\")).alias(\"or_result\"),\n",
+    "    ]\n",
+    ")"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "25dede69",
    "metadata": {},
    "source": [
-    "Quite often, it's useful to have a count of the number of true values. If you take the sum of boolean columns in a **pandas** data frame, it will tot up the number of `True` values:"
+    "Quite often, it's useful to have a count of the number of true values. If you take the sum of boolean columns in a **polars** data frame, it will tot up the number of `True` values:"
    ]
   },
   {
@@ -539,7 +548,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df.sum()"
+    "df.select(pl.col(\"*\").sum())"
    ]
   },
   {
@@ -557,8 +566,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df = pd.DataFrame.from_dict({\"bool_col\": [0, 1, 0, 1, 1]})\n",
-    "df[\"bool_col\"].astype(bool)"
+    "df = pl.DataFrame({\"bool_col\": [0, 1, 0, 1, 1]})\n",
+    "df.with_columns(pl.col(\"bool_col\").cast(pl.Boolean))"
    ]
   },
   {
@@ -579,7 +588,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds = pd.read_csv(\n",
+    "diamonds = pl.read_csv(\n",
     "    \"https://github.com/mwaskom/seaborn-data/raw/master/diamonds.csv\"\n",
     ")\n",
     "diamonds.head()"
@@ -596,20 +605,20 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "7a27f0a0",
+   "id": "6b739dc5",
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds[\"expensive\"] = diamonds[\"price\"] > 1000\n",
-    "diamonds.sample(10)"
+    "diamonds = diamonds.with_columns((pl.col(\"price\") > 1000).alias(\"expensive\"))\n",
+    "diamonds.select([\"cut\", \"carat\", \"color\", \"clarity\", \"price\", \"expensive\"]).head()"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "8383cab5",
    "metadata": {},
    "source": [
-    "Of course, this could also have been achieved in a call to assign:"
+    "Of course, this could also have been achieved with `select()` or `with_columns()`:"
    ]
   },
   {
@@ -619,15 +628,18 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds.assign(expensive=lambda x: x[\"price\"] > 1000).head()"
+    "diamonds.select(\n",
+    "    [\"cut\", \"carat\", \"color\", \"clarity\", \"price\"],\n",
+    "    (pl.col(\"price\") > 1000).alias(\"expensive\"),\n",
+    ").head()"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "622ff486",
    "metadata": {},
    "source": [
-    "Another use of booleans that is quite useful when it comes to data frames is the `.isin()` function. For example, if you just want some True or False values for whether a set of columns is in a data frame:"
+    "Another use of booleans in data frames is checking if a set of column names exists. We can use a Python list comprehension with the Python `in` operator on `diamonds.columns`:"
    ]
   },
   {
@@ -637,7 +649,26 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds.columns.isin([\"x\", \"y\", \"z\"])"
+    "cols_to_check = [\"x\", \"y\", \"z\"]\n",
+    "[col in diamonds.columns for col in cols_to_check]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "polars_is_in_markdown",
+   "metadata": {},
+   "source": [
+    "Alternatively, you can check column existence natively in **polars** using `pl.Series.is_in()`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "polars_is_in_code",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pl.Series(diamonds.columns).is_in(cols_to_check)"
    ]
   },
   {
@@ -647,7 +678,7 @@
    "source": [
     "### any() and all() in data frames\n",
     "\n",
-    "A **pandas** column of booleans behaves a lot like a list of booleans, and we can apply the same logic to it via **pandas** built-in `.any()` and `.all()` methods. We expect some entries for `\"expensive\"` to be true, so `any()` should return true:"
+    "A **polars** column of booleans behaves a lot like a list of booleans, and we can apply the same logic to it via **polars** built-in `.any()` and `.all()` methods. We expect some entries for `\"expensive\"` to be true, so `any()` should return true:"
    ]
   },
   {
@@ -657,7 +688,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds[\"expensive\"].any()"
+    "diamonds.select(pl.col(\"expensive\").any())"
    ]
   },
   {
@@ -677,22 +708,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diamonds[diamonds[\"x\"] > diamonds[\"y\"]]"
+    "diamonds.filter(pl.col(\"x\") > pl.col(\"y\"))"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "9b5906ba",
    "metadata": {},
    "source": [
-    "The expression `diamonds[\"x\"] > diamonds[\"y\"]` creates a column of booleans that is used to filter to just the rows where the condition is true."
+    "The expression `pl.col(\"x\") > pl.col(\"y\")` creates a column of booleans that is used to filter to just the rows where the condition is true."
    ]
   }
  ],
  "metadata": {
-  "interpreter": {
-   "hash": "9d7534ecd9fbc7d385378f8400cf4d6cb9c6175408a574f1c99c5269f08771cc"
-  },
   "jupytext": {
    "cell_metadata_filter": "-all",
    "encoding": "# -*- coding: utf-8 -*-",