Access locations and times of Siberian Crane encounters

For this challenge, you will use a database called the Global Biodiversity Information Facility (GBIF). GBIF is compiled from species observation data all over the world, and includes everything from museum specimens to photos taken by citizen scientists in their backyards.

Explore GBIF: Before your get started, go to the GBIF occurrences search page and explore the data.

See also:

Contribute to open data

You can get your own observations added to GBIF using iNaturalist!

Set up your code to prepare for download

We will be getting data from a source called GBIF (Global Biodiversity Information Facility). We need a package called pygbif to access the data, which may not be included in your environment. Install it by running the cell below:

Code 
conda list pygbif
Code 
%pip install -q -e ..
Code 
from landmapyr.initial import create_data_dir, robust_code
from landmapyr.gbif import gbif_credentials, gbif_species_key
from landmapyr.gbif import download_gbif, load_gbif, gbif_monthly
from landmapyr.gbif import ecoregions, join_ecoregions_monthly
from landmapyr.gbif import count_by_ecoregions
from landmapyr.gbif import simplify_ecoregions_gdf, join_occurrence

Import packages: In the imports cell, we’ve included some packages that you will need. Add imports for packages that will help you:

  • Work with reproducible file paths
  • Work with tabular data
Code 
robust_code()
data_dir = create_data_dir('species')
gbif_dir = create_data_dir('species/gbif_siberian')
gbif_dir
'/Users/brianyandell/earth-analytics/data/species/gbif_siberian'

Register and log in to GBIF

You will need a GBIF account to complete this challenge. You can use your GitHub account to authenticate with GBIF. Then, run the following code to save your credentials on your computer.

Warning

Your email address must match the email you used to sign up for GBIF!

Tip

If you accidentally enter your credentials wrong, you can set reset_credentials=True instead of reset_credentials=False. Look to top of screen for entry of credentials.

Code 
gbif_credentials(False)

Get the species key

** Your task**

  1. Replace the species_name with the name of the species you want to look up
  2. Run the code to get the species key
Code 
species_name, species_key = gbif_species_key('grus leucogeranus')
species_name, species_key
('Grus leucogeranus', 2474961)

Download data from GBIF

Code 
gbif_path = download_gbif(gbif_dir, species_key, year=None)
gbif_path
'/Users/brianyandell/earth-analytics/data/species/gbif_siberian/0001177-250227182400228.zip'
INFO:Your download key is 0001177-250227182400228
INFO:Download file size: 171492 bytes
INFO:On disk at /Users/brianyandell/earth-analytics/data/species/gbif_siberian/0001177-250227182400228.zip

Load the GBIF data into Python

Load GBIF data:

  • Look at the beginning of the file you downloaded using the code below. What do you think the delimiter is?
  • Run the following code cell. What happens?
  • Uncomment and modify the parameters of pd.read_csv() below until your data loads successfully and you have only the columns you want.

You can use the following code to look at the beginning of your file:

I copied from Lauren Alexandra and Lauren Gleason

Code 
gbif_df = load_gbif(gbif_path)
print(gbif_df.head())
          countryCode stateProvince  decimalLatitude  decimalLongitude  month  \
gbifID                                                                          
985829831          IN     Rajasthan        27.161905         77.522800    2.0   
979229641          CN       Jiangxi        28.870571        116.433170   11.0   
978902062          IR    Mazandaran        36.667110         52.550186   11.0   
978782158          IN     Rajasthan        27.161905         77.522800    1.0   
977810003          IN     Rajasthan        27.161905         77.522800    1.0   

             year  
gbifID             
985829831  1991.0  
979229641  1988.0  
978902062  2011.0  
978782158  1991.0  
977810003  1992.0

Convert GBIF data to a GeoDataFrame by Month

Code 
monthly_gdf = gbif_monthly(gbif_df)
monthly_gdf
year month geometry
gbifID
985829831 1991.0 2.0 POINT (77.5228 27.1619)
979229641 1988.0 11.0 POINT (116.43317 28.87057)
978902062 2011.0 11.0 POINT (52.55019 36.66711)
978782158 1991.0 1.0 POINT (77.5228 27.1619)
977810003 1992.0 1.0 POINT (77.5228 27.1619)
... ... ... ...
1019036144 1983.0 6.0 POINT (-90 43.75)
1019036117 1983.0 6.0 POINT (-90 43.75)
1019036092 1983.0 6.0 POINT (-90 43.75)
1019036069 1983.0 6.0 POINT (-90 43.75)
1019035937 1983.0 6.0 POINT (-90 43.75)

2936 rows × 3 columns

Download and save ecoregion boundaries

Ecoregions represent boundaries formed by biotic and abiotic conditions: geology, landforms, soils, vegetation, land use, wildlife, climate, and hydrology.

Code 
ecoregions_gdf = ecoregions(data_dir)
ecoregions_gdf.plot(edgecolor='black', color='skyblue')
Figure 1
Code 
%%bash
find ~/earth-analytics/data/species -name '*.shp'
Code 
%store ecoregions_gdf monthly_gdf
Stored 'ecoregions_gdf' (GeoDataFrame)
Stored 'monthly_gdf' (GeoDataFrame)

Identify the ecoregion for each observation

Code 
gbif_ecoregion_gdf = join_ecoregions_monthly(ecoregions_gdf, monthly_gdf)
gbif_ecoregion_gdf
year month name
ecoregion
5 2015.0 3.0 Al-Hajar foothill xeric woodlands and shrublands
5 2015.0 3.0 Al-Hajar foothill xeric woodlands and shrublands
5 2014.0 7.0 Al-Hajar foothill xeric woodlands and shrublands
5 2017.0 12.0 Al-Hajar foothill xeric woodlands and shrublands
8 NaN NaN Alashan Plateau semi-desert
... ... ... ...
802 2023.0 1.0 Yellow Sea saline meadow
802 2018.0 1.0 Yellow Sea saline meadow
802 2015.0 2.0 Yellow Sea saline meadow
802 2018.0 1.0 Yellow Sea saline meadow
802 2015.0 1.0 Yellow Sea saline meadow

2269 rows × 3 columns

Count the observations in each ecoregion each decade and month

Code 
occurrence_month_df = count_by_ecoregions(gbif_ecoregion_gdf,
                        'ecoregion', 'name', 'month')
occurrence_month_df
occurrences norm_occurrences
ecoregion month
5 3.0 2 0.098214
24 5.0 6 0.156250
9.0 2 0.142857
53 3.0 9 0.098214
74 1.0 3 0.016181
... ... ... ...
758 5.0 20 0.132275
6.0 16 0.066253
802 1.0 4 0.021575
2.0 3 0.025840
12.0 2 0.013605

78 rows × 2 columns

Code 
occurrence_decade_df = count_by_ecoregions(gbif_ecoregion_gdf,
                        'ecoregion', 'name', 'decade')
occurrence_decade_df
occurrences norm_occurrences
ecoregion decade
5 2010.0 4 0.026250
24 2010.0 4 0.030000
2020.0 3 0.013895
51 2010.0 2 0.026250
53 2020.0 9 0.016211
74 2010.0 2 0.013125
2020.0 6 0.024316
107 2000.0 4 0.005340
2010.0 53 0.028393
2020.0 90 0.029775
134 2010.0 3 0.008750
2020.0 15 0.027018
140 2010.0 2 0.021000
2020.0 3 0.019453
154 1980.0 12 0.001560
1990.0 9 0.001044
2000.0 19 0.006342
2010.0 509 0.068170
2020.0 431 0.035647
198 2010.0 19 0.033250
2020.0 11 0.011888
321 1910.0 2 0.138889
1980.0 8 0.028309
2000.0 4 0.036345
2010.0 8 0.029167
2020.0 14 0.031521
350 2010.0 2 0.005833
2020.0 16 0.028819
445 2000.0 5 0.029737
2010.0 24 0.057273
2020.0 4 0.005895
476 2010.0 24 0.026250
491 2010.0 9 0.020543
2020.0 14 0.019735
493 1990.0 12 0.030303
2000.0 2 0.014538
2010.0 13 0.037917
615 2010.0 2 0.026250
644 2010.0 83 0.026250
677 2020.0 4 0.016211
713 2000.0 8 0.035684
2010.0 18 0.032216
2020.0 18 0.019895
716 2010.0 9 0.001327
2020.0 347 0.031602
744 2010.0 2 0.026250
757 1970.0 17 0.012903
1980.0 90 0.029587
1990.0 138 0.040469
2000.0 65 0.054869
758 1980.0 47 0.037420
1990.0 17 0.012074
802 2010.0 8 0.042000
2020.0 2 0.006484
Code 
# plot to check distributions 
occurrence_decade_df.reset_index().plot.scatter(
    x='decade', y='occurrences', c='ecoregion',
    logy=True
)
Figure 2

Create a simplified GeoDataFrame for plot

Code 
ecoregions_gdf = simplify_ecoregions_gdf(ecoregions_gdf)
ecoregions_gdf
name area geometry
ecoregion
0 Adelie Land tundra 0.038948 MULTIPOLYGON EMPTY
1 Admiralty Islands lowland rain forests 0.170599 POLYGON ((16411777.375 -229101.376, 16384825.7...
2 Aegean and Western Turkey sclerophyllous and m... 13.844952 MULTIPOLYGON (((3391149.749 4336064.109, 33846...
3 Afghan Mountains semi-desert 1.355536 MULTIPOLYGON (((7369001.698 4093509.259, 73168...
4 Ahklun and Kilbuck Upland Tundra 8.196573 MULTIPOLYGON (((-17930832.005 8046779.358, -17...
... ... ... ...
842 Sulawesi lowland rain forests 9.422097 MULTIPOLYGON (((14113374.546 501721.962, 14128...
843 East African montane forests 5.010930 MULTIPOLYGON (((4298787.669 -137583.786, 42727...
844 Eastern Arc forests 0.890325 MULTIPOLYGON (((4267432.68 -493759.165, 428533...
845 Borneo montane rain forests 9.358407 MULTIPOLYGON (((13126956.393 539092.917, 13136...
846 Kinabalu montane alpine meadows 0.352694 POLYGON ((12981819.186 696445.445, 12997053.80...

847 rows × 3 columns

Mapping decadal distribution

Code 
occurrence_gdf = join_occurrence(ecoregions_gdf, occurrence_decade_df)
occurrence_gdf
name area geometry norm_occurrences
ecoregion decade
5 2010.0 Al-Hajar foothill xeric woodlands and shrublands 4.099668 POLYGON ((6264504.021 2842331.306, 6336024.085... 0.026250
24 2010.0 Amur meadow steppe 15.118769 MULTIPOLYGON (((15067649.194 6001589.024, 1503... 0.030000
2020.0 Amur meadow steppe 15.118769 MULTIPOLYGON (((15067649.194 6001589.024, 1503... 0.013895
51 2010.0 European Atlantic mixed forests 48.383344 MULTIPOLYGON (((-509089.943 6127087.807, -5328... 0.026250
53 2020.0 Azerbaijan shrub desert and steppe 6.794797 POLYGON ((5427403.54 5089371.081, 5512543.361 ... 0.016211
74 2010.0 Bohai Sea saline meadow 1.187602 MULTIPOLYGON (((13116898.113 4589071.254, 1319... 0.013125
2020.0 Bohai Sea saline meadow 1.187602 MULTIPOLYGON (((13116898.113 4589071.254, 1319... 0.024316
107 2000.0 Caspian Hyrcanian mixed forests 5.604168 MULTIPOLYGON (((5333143.547 4574764.516, 52538... 0.005340
2010.0 Caspian Hyrcanian mixed forests 5.604168 MULTIPOLYGON (((5333143.547 4574764.516, 52538... 0.028393
2020.0 Caspian Hyrcanian mixed forests 5.604168 MULTIPOLYGON (((5333143.547 4574764.516, 52538... 0.029775
134 2010.0 Central China Loess Plateau mixed forests 37.104395 MULTIPOLYGON (((13378649.084 5076474.672, 1331... 0.008750
2020.0 Central China Loess Plateau mixed forests 37.104395 MULTIPOLYGON (((13378649.084 5076474.672, 1331... 0.027018
140 2010.0 Central Korean deciduous forests 10.616297 MULTIPOLYGON (((14084109.316 4498884.57, 14068... 0.021000
2020.0 Central Korean deciduous forests 10.616297 MULTIPOLYGON (((14084109.316 4498884.57, 14068... 0.019453
154 1980.0 Changjiang Plain evergreen forests 41.203493 MULTIPOLYGON (((13613475.56 3473108.645, 13581... 0.001560
1990.0 Changjiang Plain evergreen forests 41.203493 MULTIPOLYGON (((13613475.56 3473108.645, 13581... 0.001044
2000.0 Changjiang Plain evergreen forests 41.203493 MULTIPOLYGON (((13613475.56 3473108.645, 13581... 0.006342
2010.0 Changjiang Plain evergreen forests 41.203493 MULTIPOLYGON (((13613475.56 3473108.645, 13581... 0.068170
2020.0 Changjiang Plain evergreen forests 41.203493 MULTIPOLYGON (((13613475.56 3473108.645, 13581... 0.035647
198 2010.0 Daurian forest steppe 26.252795 POLYGON ((13320358.199 6772264.491, 13349680.3... 0.033250
2020.0 Daurian forest steppe 26.252795 POLYGON ((13320358.199 6772264.491, 13349680.3... 0.011888
321 1910.0 Huang He Plain mixed forests 43.295555 POLYGON ((13332332.499 4862491.773, 13335344.1... 0.138889
1980.0 Huang He Plain mixed forests 43.295555 POLYGON ((13332332.499 4862491.773, 13335344.1... 0.028309
2000.0 Huang He Plain mixed forests 43.295555 POLYGON ((13332332.499 4862491.773, 13335344.1... 0.036345
2010.0 Huang He Plain mixed forests 43.295555 POLYGON ((13332332.499 4862491.773, 13335344.1... 0.029167
2020.0 Huang He Plain mixed forests 43.295555 POLYGON ((13332332.499 4862491.773, 13335344.1... 0.031521
350 2010.0 Jian Nan subtropical evergreen forests 59.745023 MULTIPOLYGON (((13366375.518 3078256.552, 1334... 0.005833
2020.0 Jian Nan subtropical evergreen forests 59.745023 MULTIPOLYGON (((13366375.518 3078256.552, 1334... 0.028819
445 2000.0 Mongolian-Manchurian grassland 103.136132 POLYGON ((13399045.827 6690406.546, 13411644.4... 0.029737
2010.0 Mongolian-Manchurian grassland 103.136132 POLYGON ((13399045.827 6690406.546, 13411644.4... 0.057273
2020.0 Mongolian-Manchurian grassland 103.136132 POLYGON ((13399045.827 6690406.546, 13411644.4... 0.005895
476 2010.0 Nihonkai evergreen forests 2.172062 MULTIPOLYGON (((14838329.599 4319227.511, 1484... 0.026250
491 2010.0 Northeast China Plain deciduous forests 26.069601 MULTIPOLYGON (((13285446.855 5177582.449, 1327... 0.020543
2020.0 Northeast China Plain deciduous forests 26.069601 MULTIPOLYGON (((13285446.855 5177582.449, 1327... 0.019735
493 1990.0 Northeast Siberian coastal tundra 54.583079 MULTIPOLYGON (((17977352.099 10838116.421, 179... 0.030303
2000.0 Northeast Siberian coastal tundra 54.583079 MULTIPOLYGON (((17977352.099 10838116.421, 179... 0.014538
2010.0 Northeast Siberian coastal tundra 54.583079 MULTIPOLYGON (((17977352.099 10838116.421, 179... 0.037917
615 2010.0 Selenge-Orkhon forest steppe 28.090333 MULTIPOLYGON (((10550560.509 6345569.594, 1061... 0.026250
644 2010.0 South China-Vietnam subtropical evergreen forests 19.600028 MULTIPOLYGON (((12348762.017 2258424.919, 1235... 0.026250
677 2020.0 Southern Korea evergreen forests 1.449664 MULTIPOLYGON (((14067544.103 4086271.62, 14056... 0.016211
713 2000.0 Taiheiyo evergreen forests 13.553387 MULTIPOLYGON (((14748057.024 4038696.166, 1473... 0.035684
2010.0 Taiheiyo evergreen forests 13.553387 MULTIPOLYGON (((14748057.024 4038696.166, 1473... 0.032216
2020.0 Taiheiyo evergreen forests 13.553387 MULTIPOLYGON (((14748057.024 4038696.166, 1473... 0.019895
716 2010.0 Taiwan subtropical evergreen forests 2.953675 POLYGON ((13463176.55 2555136.408, 13437821.53... 0.001327
2020.0 Taiwan subtropical evergreen forests 2.953675 POLYGON ((13463176.55 2555136.408, 13437821.53... 0.031602
744 2010.0 Trans-Baikal conifer forests 25.732184 MULTIPOLYGON (((12494118.896 6222155.236, 1247... 0.026250
757 1970.0 Upper Gangetic Plains moist deciduous forests 23.895360 POLYGON ((8770042.323 3393768.842, 8910982.032... 0.012903
1980.0 Upper Gangetic Plains moist deciduous forests 23.895360 POLYGON ((8770042.323 3393768.842, 8910982.032... 0.029587
1990.0 Upper Gangetic Plains moist deciduous forests 23.895360 POLYGON ((8770042.323 3393768.842, 8910982.032... 0.040469
2000.0 Upper Gangetic Plains moist deciduous forests 23.895360 POLYGON ((8770042.323 3393768.842, 8910982.032... 0.054869
758 1980.0 Upper Midwest US forest-savanna transition 15.481685 MULTIPOLYGON (((-9686382.157 5638236.966, -973... 0.037420
1990.0 Upper Midwest US forest-savanna transition 15.481685 MULTIPOLYGON (((-9686382.157 5638236.966, -973... 0.012074
802 2010.0 Yellow Sea saline meadow 0.517810 POLYGON ((13451648.07 3834357.593, 13303152.21... 0.042000
2020.0 Yellow Sea saline meadow 0.517810 POLYGON ((13451648.07 3834357.593, 13303152.21... 0.006484

Static Plot

Code 
from landmapyr.plots import plot_occurrence
plot_occurrence(occurrence_gdf, 'decade', ncols=1)
Figure 3

Optional Dynamic Plot

Code 
from landmapyr.hv_plots import hvplot_occurrence
occurrence_hvplot = hvplot_occurrence(occurrence_gdf, 'decade',
  title="Grus leucogeranus Siberian Crane Decadal Tracking")
# Save the plot
occurrence_hvplot.save('siberian-crane-decades.html', embed=True)
  0%|          | 0/7 [00:00<?, ?it/s] 57%|█████▋    | 4/7 [00:00<00:00, 34.45it/s]                                             
WARNING:W-1005 (FIXED_SIZING_MODE): 'fixed' sizing mode requires width and height to be set: figure(id='1da8b71a-a35e-4f98-a085-041c0be44a19', ...)
Code 
occurrence_hvplot
<iframe src="siberian-crane-decades.html" width="100%" height="500px"
        style="border: none;"></iframe>

If not shown, see Dynamic Images from Environmental Data Analytics Projects.