From 3a9f59674f3c5ef130950325f6f66f3db2a4b043 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Thu, 19 Sep 2024 21:36:51 -0700 Subject: [PATCH 1/8] WIP chapter 8 - mapping --- chapters/08-mapping.qmd | 173 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 173 insertions(+) diff --git a/chapters/08-mapping.qmd b/chapters/08-mapping.qmd index fc67eca..9236d5d 100644 --- a/chapters/08-mapping.qmd +++ b/chapters/08-mapping.qmd @@ -7,3 +7,176 @@ project: # Making maps with Julia {#sec-map-making} ## Prerequisites {.unnumbered} + +This chapter requires the following packages: + +```{julia} +# We use the Makie.jl ecosystem primarily. +using CairoMakie +using GeoMakie +using Tyler +using NaturalEarth # Natural Earth vector data +using GeoDataFrames, ArchGDAL +import GeoInterface as GI, GeometryOps as GO, GeoFormatTypes as GFT +import LibGEOS # to activate some capabilities of GeometryOps +using FlexiJoins # dataframe joins +using Rasters +``` + +It also relies on the following data files: + +```{julia} +nz = GeoDataFrames.read("data/nz.gpkg") +nz_height = GeoDataFrames.read("data/nz_height.gpkg") +nz_elev = Raster("data/nz_elev.tif") +tanzania = filter!( + :name_long => ==("Tanzania"), + GeoDataFrames.read("data/world.gpkg") +) +tanzania_buf = GO.reproject( + GO.buffer( + GO.reproject(tanzania, GFT.EPSG(32736)), + 50000), + GFT.EPSG(4326) +) +tanzania_neigh = filter!( + :geom => x -> GO.intersects(x, only(tanzania_buf.geom)), + GeoDataFrames.read("data/world.gpkg") +) +``` + +## Introduction + +A satisfying and important aspect of geographic research is communicating the results. +Map making---the art of cartography---is an ancient skill that involves communication, intuition, and an element of creativity. +In addition to being fun and creative, cartography also has important practical applications. +A carefully crafted map can be the best way of communicating the results of your work, but poorly designed maps can leave a bad impression. +Common design issues include poor placement, size and readability of text and careless selection of colors, as outlined in the style guide of the Journal of Maps. +Furthermore, poor map making can hinder the communication of results [@brewer_designing_2015]: + +> Amateur-looking maps can undermine your audience's ability to understand important information and weaken the presentation of a professional data investigation. + +Maps have been used for several thousand years for a wide variety of purposes. +Historic examples include maps of buildings and land ownership in the Old Babylonian dynasty more than 3000 years ago and Ptolemy's world map in his masterpiece Geography nearly 2000 years ago [@talbert_ancient_2014]. + +Map making has historically been an activity undertaken only by, or on behalf of, the elite. +This has changed with the emergence of open source mapping software such as mapping packages in Python, R, and other languages, and the "print composer" in QGIS, which enable anyone to make high-quality maps, enabling "citizen science". +Maps are also often the best way to present the findings of geocomputational research in a way that is accessible. +Map making is therefore a critical part of geocomputation and its emphasis not only on describing, but also changing the world. + +Basic static display of vector layers in Julia can be done with the `plot` function from **Makie.jl**, as we saw in Sections @sec-vector-layers and @sec-using-rasterio. +Other, more advanced uses of these methods, were also encountered in subsequent chapters, when demonstrating the various outputs we got. +In this chapter, we provide a comprehensive summary of the most useful workflows of these two methods for creating static maps (@sec-static-maps). +Static maps can be easily shared and viewed (whether digitally or in print), however they can only convey as much information as a static image can. +Interactive maps provide much more flexibilty in terms of user experience and amount of information, however they often require more work to design and effectively share. +Thus, in @sec-interactive-maps, we move on to elaborate on the `.explore` method for creating interactive maps, which was also briefly introduced earlier in @sec-vector-layers. + +## Static maps {#sec-static-maps} + +Static maps are the most common type of visual output from geocomputation. +For example, we have been using static **Makie.jl** plots throughout the book, to display vector and raster data. + +In this section we systematically review and elaborate on the various properties that can be customized when using those functions. + +A static map is basically a digital image. +When stored in a file, standard formats include `.png` and `.pdf` for graphical raster and vector outputs, respectively. +Thanks to their simplicity, static maps can be shared in a wide variety of ways: in print, through files sent by e-mail, embedded in documents and web pages, etc. + +In the **Makie.jl** ecosystem, there is not much difference between static and interactive or dynamic maps, since everything can be updated on-the-fly using `Observables`. We will show how to do this shortly. + +Nevertheless, there are many aesthetic considerations when making a static map, and there is also a wide variety of ways to create static maps using novel presentation methods. +This is the focus of the field of cartography, and beyond the scope of this book. + +### Minimal examples + +A vector layer can be displayed by plotting its geometry. +A minimal example of a vector layer map is obtained using `plot` with nothing but the defaults (@fig-vector-minimal). + +```julia +#| label: fig-vector-minimal +#| fig-cap: Minimal example of a static vector layer plot with `.plot` +plot(nz.geom) +``` + +A `Raster`, or any Julia matrix for that matter, cna be displayed using any 2D plotting function, like `heatmap` or `surface`. +@fig-raster-minimal shows a minimal example of a static raster map. + +```julia +#| label: fig-raster-minimal +#| fig-cap: Minimal example of a static raster plot with `heatmap` +heatmap(nz_elev) +``` + +### Styling {#sec-static-styling} + +The most useful visual properties of the geometries, that can be specified in plotting functions, include `color`, `strokecolor`, and `markersize` (for points) (@fig-basic-plot). + +```{python} +#| label: fig-basic-plot +#| fig-cap: Setting `color` and `edgecolor` in static maps of a vector layer +#| fig-subcap: +#| - Light grey fill +#| - No fill, blue edge +#| - Light grey fill, blue edge +#| layout-ncol: 3 +display(poly(nz.geom; color=:lightgrey)) +display(poly(nz.geom; color=:transparent, strokecolor=:blue, strokewidth=1)) +display(poly(nz.geom; color=:lightgrey, strokecolor=:blue, strokewidth=1)) +``` + +The next example uses `markersize` to get larger points (@fig-basic-plot-markersize). +It also demonstrates how to control the overall figure size, such as $1000 \times 1000$ $px$ in this case, using the explicit `Figure` constructor to initialize the plot and its `size` parameter to specify size. + +```{julia} +#| label: fig-basic-plot-markersize +#| fig-cap: Setting `markersize` in a static map of a vector layer +fig = Figure(size=(4 * 72,4 * 72)) +ax, plt = scatter(fig[1, 1], nz_height.geom, markersize=100) +fig +``` + + +::: callout-note +As you have probably noticed throughout the book, the `plt.subplots` function is used to initialize a **maptplotlib** plot layout, possibly also specifying image size (e.g., @fig-basic-plot-markersize) and multi-panel layout (e.g., @fig-faceted-map). +The returned value is a `tuple` of `Figure` and `Axes` objects, which we conventionally unpack to variables named `fig` and `ax`. +These two variables represent the entire figure, and the elements of individual sub-figures, respectively. + +For our purposes in this book, we have been using just the `ax` object, passing it to the `ax` parameter in further function calls, in order to add subsequent layers (e.g., @fig-plot-raster-and-vector) or other elements (e.g., @fig-plot-symbology-colors-r-scale) into the same panel. +In a single-panel figure, we pass `ax` itself, whereas in a multi-panel figure we pass individual elements representing a specific panel (such as `ax[0]` or `ax[0][0]`, depending of the layout; see @sec-faceted-maps) + +Note that in some of the cases we have used an alternative to `plt.subplots`---we assigned an initial plot into a variable, conventionally named `base`, similarly passing it to the `ax` parameter of further calls, e.g., to add subsequent layers (e.g., @fig-two-layers); this (shorter) syntax, though, is less general than `plt.subplots` and not applicable in some of the cases (such as displaying a raster and a vector layer in the same plot, e.g., @fig-plot-raster-and-vector). + +TODO: translate this to Julia. +::: + +### Symbology {#sec-plot-symbology} + +We consider the following concepts to be "symbology": +- Legend +- Color +- Color map + +For example, @fig-plot-symbology shows the `nz` polygons colored according to the `'Median_income'` attribute (column), with a legend. + +::: {.panel-tabset} + +## Pure Makie.jl +```{julia} +#| label: fig-plot-symbology +#| fig-cap: Symbology in a static map created with `poly` +fig, ax, plt = poly(nz.geom; color = nz.Median_income) +cb = Colorbar(fig[1, 2], plt; label = "Median income") +fig +``` + +## AlgebraOfGraphics.jl + +```{julia} +#| fig-cap: Symbology in a static map created with AlgebraOfGraphics.jl +using AlgebraOfGraphics +data(nz) * mapping(:geometry; color = :Median_income) * visual(Poly) |> draw +``` +::: + +The default color map which you see in @fig-plot-symbology is `colormap=:viridis`. +The `colormap` argument can be used to specify one of countless color scales. From fe81d7bf0e5a2401f87b8a038075d594f34ddaf8 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Fri, 20 Sep 2024 17:21:27 -0700 Subject: [PATCH 2/8] Add AoG and Tyler to Project.toml --- Project.toml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Project.toml b/Project.toml index d3f0dfa..edb0c32 100644 --- a/Project.toml +++ b/Project.toml @@ -1,4 +1,5 @@ [deps] +AlgebraOfGraphics = "cbdf2221-f076-402e-a563-3d30da359d67" ArchGDAL = "c9ce4bd3-c3d5-55b8-8973-c0e20141b8c3" CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0" DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" @@ -14,6 +15,7 @@ Proj = "c94c279d-25a6-4763-9509-64d165bea63e" Rasters = "a3a2b9e3-a471-40c9-b274-f788e487c689" Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91" +Tyler = "e170d443-b9d6-418a-9ee8-061e966341ef" [compat] GeometryOps = "0.1.12" From ed41fc5b5d05148eae3d17ea93f1ce404bbf2e83 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Fri, 20 Sep 2024 18:39:07 -0700 Subject: [PATCH 3/8] add NaturalEarth.jl --- Project.toml | 1 + 1 file changed, 1 insertion(+) diff --git a/Project.toml b/Project.toml index edb0c32..0951a99 100644 --- a/Project.toml +++ b/Project.toml @@ -11,6 +11,7 @@ GeoJSON = "61d90e0f-e114-555e-ac52-39dfb47a3ef9" GeoMakie = "db073c08-6b98-4ee5-b6a4-5efafb3259c6" GeometryOps = "3251bfac-6a57-4b6d-aa61-ac1fef2975ab" LibGEOS = "a90b1aa1-3769-5649-ba7e-abc5a9d163eb" +NaturalEarth = "436b0209-26ab-4e65-94a9-6526d86fea76" Proj = "c94c279d-25a6-4763-9509-64d165bea63e" Rasters = "a3a2b9e3-a471-40c9-b274-f788e487c689" Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2" From 840707e864b106ae0570321ac968ef099c2b5faf Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Sat, 21 Sep 2024 10:45:38 -0700 Subject: [PATCH 4/8] Get things building --- Project.toml | 3 +++ chapters/08-mapping.qmd | 11 ++++++----- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Project.toml b/Project.toml index 0951a99..1c306d4 100644 --- a/Project.toml +++ b/Project.toml @@ -4,6 +4,7 @@ ArchGDAL = "c9ce4bd3-c3d5-55b8-8973-c0e20141b8c3" CairoMakie = "13f3f980-e62b-5c42-98c6-ff1f3baf88f0" DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0" DimensionalData = "0703355e-b756-11e9-17c0-8b28908087d0" +FlexiJoins = "e37f2e79-19fa-4eb7-8510-b63b51fe0a37" GeoDataFrames = "62cb38b5-d8d2-4862-a48e-6a340996859f" GeoFormatTypes = "68eda718-8dee-11e9-39e7-89f7f65f511f" GeoInterface = "cf35fbd7-0cd7-5166-be24-54bfbe79505f" @@ -11,6 +12,8 @@ GeoJSON = "61d90e0f-e114-555e-ac52-39dfb47a3ef9" GeoMakie = "db073c08-6b98-4ee5-b6a4-5efafb3259c6" GeometryOps = "3251bfac-6a57-4b6d-aa61-ac1fef2975ab" LibGEOS = "a90b1aa1-3769-5649-ba7e-abc5a9d163eb" +Makie = "ee78f7c6-11fb-53f2-987a-cfe4a2b5a57a" +MakieCore = "20f20a25-4f0e-4fdf-b5d1-57303727442b" NaturalEarth = "436b0209-26ab-4e65-94a9-6526d86fea76" Proj = "c94c279d-25a6-4763-9509-64d165bea63e" Rasters = "a3a2b9e3-a471-40c9-b274-f788e487c689" diff --git a/chapters/08-mapping.qmd b/chapters/08-mapping.qmd index 9236d5d..9b5f0d8 100644 --- a/chapters/08-mapping.qmd +++ b/chapters/08-mapping.qmd @@ -35,8 +35,9 @@ tanzania = filter!( ) tanzania_buf = GO.reproject( GO.buffer( - GO.reproject(tanzania, GFT.EPSG(32736)), + GO.reproject(tanzania, GI.crs(tanzania), GFT.EPSG(32736)), 50000), + GFT.EPSG(32736), GFT.EPSG(4326) ) tanzania_neigh = filter!( @@ -92,7 +93,7 @@ This is the focus of the field of cartography, and beyond the scope of this book A vector layer can be displayed by plotting its geometry. A minimal example of a vector layer map is obtained using `plot` with nothing but the defaults (@fig-vector-minimal). -```julia +```{julia} #| label: fig-vector-minimal #| fig-cap: Minimal example of a static vector layer plot with `.plot` plot(nz.geom) @@ -101,7 +102,7 @@ plot(nz.geom) A `Raster`, or any Julia matrix for that matter, cna be displayed using any 2D plotting function, like `heatmap` or `surface`. @fig-raster-minimal shows a minimal example of a static raster map. -```julia +```{julia} #| label: fig-raster-minimal #| fig-cap: Minimal example of a static raster plot with `heatmap` heatmap(nz_elev) @@ -111,7 +112,7 @@ heatmap(nz_elev) The most useful visual properties of the geometries, that can be specified in plotting functions, include `color`, `strokecolor`, and `markersize` (for points) (@fig-basic-plot). -```{python} +```{julia} #| label: fig-basic-plot #| fig-cap: Setting `color` and `edgecolor` in static maps of a vector layer #| fig-subcap: @@ -174,7 +175,7 @@ fig ```{julia} #| fig-cap: Symbology in a static map created with AlgebraOfGraphics.jl using AlgebraOfGraphics -data(nz) * mapping(:geometry; color = :Median_income) * visual(Poly) |> draw +data(nz) * mapping(:geom; color = :Median_income) * visual(Poly) |> draw ``` ::: From 8d7da19ffb13c3121fd1afa3547c311ac5ac5a59 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Sun, 22 Sep 2024 11:45:31 -0700 Subject: [PATCH 5/8] Split up the loading code into loading and generating sections --- chapters/08-mapping.qmd | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/chapters/08-mapping.qmd b/chapters/08-mapping.qmd index 9b5f0d8..f9fabdd 100644 --- a/chapters/08-mapping.qmd +++ b/chapters/08-mapping.qmd @@ -17,7 +17,9 @@ using GeoMakie using Tyler using NaturalEarth # Natural Earth vector data using GeoDataFrames, ArchGDAL -import GeoInterface as GI, GeometryOps as GO, GeoFormatTypes as GFT +import GeoInterface as GI +import GeometryOps as GO +import GeoFormatTypes as GFT import LibGEOS # to activate some capabilities of GeometryOps using FlexiJoins # dataframe joins using Rasters @@ -33,6 +35,12 @@ tanzania = filter!( :name_long => ==("Tanzania"), GeoDataFrames.read("data/world.gpkg") ) +``` + +From the data loaded above, we can create some more geometries. +First, we create a buffer around the `tanzania` polygon. In order to create this buffer, we need to be in a Cartesian ("planar") CRS, and the Tanzania polygon is in lat/long space (which is "geographic" or "ellipsoidal"). +Thus, we first reproject the polygon into a Cartesian CRS, create the buffer, and then reproject back into lat/long space. +```{julia} tanzania_buf = GO.reproject( GO.buffer( GO.reproject(tanzania, GI.crs(tanzania), GFT.EPSG(32736)), @@ -40,6 +48,9 @@ tanzania_buf = GO.reproject( GFT.EPSG(32736), GFT.EPSG(4326) ) +``` +We can also create a dataframe of all of Tanzania's neighbours: +```{julia} tanzania_neigh = filter!( :geom => x -> GO.intersects(x, only(tanzania_buf.geom)), GeoDataFrames.read("data/world.gpkg") @@ -180,4 +191,6 @@ data(nz) * mapping(:geom; color = :Median_income) * visual(Poly) |> draw ::: The default color map which you see in @fig-plot-symbology is `colormap=:viridis`. -The `colormap` argument can be used to specify one of countless color scales. +The `colormap` argument can be used to specify one of countless color schemes - you can manually construct your own, or use one of the [many available color schemes in Makie](https://docs.makie.org/stable/explanations/colors). + + From 3b3478590e3372dd55d222ec7ec9c018ef8b78df Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Sun, 22 Sep 2024 12:26:11 -0700 Subject: [PATCH 6/8] Move env var definition to global scope --- .github/workflows/pr.yaml | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/.github/workflows/pr.yaml b/.github/workflows/pr.yaml index e170eb2..206d325 100644 --- a/.github/workflows/pr.yaml +++ b/.github/workflows/pr.yaml @@ -13,6 +13,8 @@ jobs: statuses: write pull-requests: write deployments: write + env: + QUARTO_JULIA_PROJECT: "@quarto" # this ensures we use the custom version of QuartoNotebookRunner.jl steps: - uses: actions/checkout@v3 - uses: julia-actions/setup-julia@v2 @@ -37,15 +39,10 @@ jobs: - name: Set up Quarto uses: quarto-dev/quarto-actions/setup@v2 - env: - QUARTO_JULIA_PROJECT: "@quarto" - - name: Render uses: quarto-dev/quarto-actions/render@v2 with: to: html - env: - QUARTO_JULIA_PROJECT: "@quarto" - name: Deploy Preview to Netlify as preview id: netlify-deploy From 60739935c311a5a347afa75397abf31c26b722a9 Mon Sep 17 00:00:00 2001 From: Anshul Singhvi Date: Sun, 22 Sep 2024 12:26:41 -0700 Subject: [PATCH 7/8] Move env var to toplevel --- .github/workflows/main_new.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/main_new.yaml b/.github/workflows/main_new.yaml index de9bc80..4e90101 100644 --- a/.github/workflows/main_new.yaml +++ b/.github/workflows/main_new.yaml @@ -11,6 +11,8 @@ jobs: permissions: contents: write statuses: write + env: + QUARTO_JULIA_PROJECT: "@quarto" steps: - uses: actions/checkout@v3 - uses: julia-actions/setup-julia@v2 @@ -35,11 +37,9 @@ jobs: - name: Set up Quarto uses: quarto-dev/quarto-actions/setup@v2 - env: - QUARTO_JULIA_PROJECT: "@quarto" - name: Render and Publish uses: quarto-dev/quarto-actions/publish@v2 with: target: netlify - NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }} \ No newline at end of file + NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }} From 12a3efaa7598cc2412dab9b9a9f20f2a6e380635 Mon Sep 17 00:00:00 2001 From: Jakub Nowosad Date: Mon, 23 Sep 2024 08:19:26 +0000 Subject: [PATCH 8/8] fixing data path --- chapters/01-spatial-data.qmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/chapters/01-spatial-data.qmd b/chapters/01-spatial-data.qmd index dac29fa..44af128 100644 --- a/chapters/01-spatial-data.qmd +++ b/chapters/01-spatial-data.qmd @@ -20,7 +20,7 @@ mkpath("output") ## Introduction ```{julia} using GeoDataFrames -df = GeoDataFrames.read("../data/world.gpkg") +df = GeoDataFrames.read("data/world.gpkg") ``` ```{julia}