{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "ae136e42", "metadata": {}, "source": [ "# ECMWF Open Data\n", "\n", "This demonstrates how to use the ECMWF Real-Time Open Data\n", "\n", "> [Media Statement](https://www.ecmwf.int/en/about/media-centre/news/2022/ecmwf-makes-wide-range-data-openly-available)\n", ">\n", "> The data that are becoming available are based on a range of high-resolution forecasts (HRES – 9 km horizontal resolution) and ensemble forecasts (ENS – 18 km horizontal resolution). **They will be made accessible at a resolution of 0.4 x 0.4 degrees**.\n", "\n", "This data is freely available from ECMWF in GRIB2 format ([read more 👀](https://confluence.ecmwf.int/display/UDOC/ECMWF+Open+Data+-+Real+Time)).\n", "\n", "There following prodcuts are available:\n", "- `\"oper\"` - operational high-resolution forecast, atmospheric fields\n", "- `\"enfo\"` - ensemble forecast, atmospheric fields\n", "- `\"wave\"` - wave forecasts\n", "- `\"waef\"` - ensemble forecast, ocean wave fields\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "b51a1150", "metadata": {}, "outputs": [], "source": [ "import cartopy.crs as ccrs\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "from herbie import Herbie\n", "\n", "from paint.standard2 import cm_tmp, cm_wind, cm_wave_height\n", "from toolbox import EasyMap, pc" ] }, { "cell_type": "code", "execution_count": 2, "id": "b6a467e6", "metadata": { "notebookRunGroups": { "groupValue": "" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Found ┊ model=ecmwf ┊ \u001b[3mproduct=oper\u001b[0m ┊ \u001b[38;2;41;130;13m2022-Jan-26 00:00 UTC\u001b[92m F12\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mGRIB2 @ local\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mIDX @ azure\u001b[0m\n" ] } ], "source": [ "H = Herbie(\"2022-01-26\", model=\"ecmwf\", product=\"oper\", fxx=12)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f4782e73", "metadata": {}, "source": [ "## Unique Index Files\n", "\n", "The ECMWF index files are different than the wgrib2-style index files, so pay close attention to how you should select the field you want." ] }, { "cell_type": "code", "execution_count": 3, "id": "9d4f1633", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Use regular expression to search for lines in the index file.\n", "Here are some examples you can use for the ecCodes-style `searchString`\n", "\n", "Look at the ECMWF GRIB Parameter Database\n", "https://apps.ecmwf.int/codes/grib/param-db\n", "\n", "======================== ==============================================\n", "searchString (oper/enso) Messages that will be downloaded\n", "======================== ==============================================\n", "\":2t:\" 2-m temperature\n", "\":10u:\" 10-m u wind vector\n", "\":10v:\" 10-m v wind vector\n", "\":10(u|v): **10m u and 10m v wind**\n", "\":d:\" Divergence (all levels)\n", "\":gh:\" geopotential height (all levels)\n", "\":gh:500\" geopotential height only at 500 hPa\n", "\":st:\" soil temperature\n", "\":tp:\" total precipitation\n", "\":msl:\" mean sea level pressure\n", "\":q:\" Specific Humidity\n", "\":r:\" relative humidity\n", "\":ro:\" Runn-off\n", "\":skt:\" skin temperature\n", "\":sp:\" surface pressure\n", "\":t:\" temperature\n", "\":tcwv:\" Total column vertically integrated water vapor\n", "\":vo:\" Relative vorticity\n", "\":v:\" v wind vector\n", "\":u:\" u wind vector\n", "\":(t|u|v|r):\" Temp, u/v wind, RH (all levels)\n", "\":500:\" All variables on the 500 hPa level\n", "\n", "======================== ==============================================\n", "searchString (wave/waef) Messages that will be downloaded\n", "======================== ==============================================\n", "\":swh:\" Significant height of wind waves + swell\n", "\":mwp:\" Mean wave period\n", "\":mwd:\" Mean wave direction\n", "\":pp1d:\" Peak wave period\n", "\":mp2:\" Mean zero-crossing wave period\n", "\n", "If you need help with regular expression, search the web or look at\n", "this cheatsheet: https://www.petefreitag.com/cheatsheets/regex/.\n", "\n" ] } ], "source": [ "# Show the searchString_help\n", "print(H.searchString_help)" ] }, { "cell_type": "code", "execution_count": 4, "id": "66a65101", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | grib_message | \n", "start_byte | \n", "end_byte | \n", "range | \n", "reference_time | \n", "valid_time | \n", "step | \n", "param | \n", "levelist | \n", "levtype | \n", "number | \n", "domain | \n", "expver | \n", "class | \n", "type | \n", "stream | \n", "search_this | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "1 | \n", "0 | \n", "609046 | \n", "0-609046 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "skt | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":skt:sfc:g:0001:od:fc:oper | \n", "
| 1 | \n", "2 | \n", "609046 | \n", "1218092 | \n", "609046-1218092 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "st | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":st:sfc:g:0001:od:fc:oper | \n", "
| 2 | \n", "3 | \n", "1218092 | \n", "1827138 | \n", "1218092-1827138 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "gh | \n", "1000 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":gh:1000:pl:g:0001:od:fc:oper | \n", "
| 3 | \n", "4 | \n", "1827138 | \n", "2436184 | \n", "1827138-2436184 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "gh | \n", "925 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":gh:925:pl:g:0001:od:fc:oper | \n", "
| 4 | \n", "5 | \n", "2436184 | \n", "2842280 | \n", "2436184-2842280 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "r | \n", "300 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":r:300:pl:g:0001:od:fc:oper | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 77 | \n", "78 | \n", "48114266 | \n", "48926262 | \n", "48114266-48926262 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "d | \n", "250 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":d:250:pl:g:0001:od:fc:oper | \n", "
| 78 | \n", "79 | \n", "48926262 | \n", "49738258 | \n", "48926262-49738258 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "vo | \n", "250 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":vo:250:pl:g:0001:od:fc:oper | \n", "
| 79 | \n", "80 | \n", "49738258 | \n", "50550254 | \n", "49738258-50550254 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "vo | \n", "50 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":vo:50:pl:g:0001:od:fc:oper | \n", "
| 80 | \n", "81 | \n", "50550254 | \n", "51159324 | \n", "50550254-51159324 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "ro | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":ro:sfc:g:0001:od:fc:oper | \n", "
| 81 | \n", "82 | \n", "51159324 | \n", "51971320 | \n", "51159324-51971320 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "d | \n", "50 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":d:50:pl:g:0001:od:fc:oper | \n", "
82 rows × 17 columns
\n", "<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 12:00:00\n",
" heightAboveGround float64 2.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] ...\n",
"Data variables:\n",
" t2m (latitude, longitude) float32 ...\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bf1...\n",
" searchString: :2t:<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" heightAboveGround float64 10.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" u10 (latitude, longitude) float32 8.174 8.174 ... -5.998\n",
" v10 (latitude, longitude) float32 3.499 3.499 ... 3.733\n",
" gribfile_projection object None\n",
" spd (latitude, longitude) float32 8.891 8.891 ... 7.065\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bfe...\n",
" searchString: :10(u|v):<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" isobaricInhPa float64 500.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" q (latitude, longitude) float32 6.679e-05 ... 0.0001641\n",
" gh (latitude, longitude) float32 5.029e+03 ... 5.136e+03\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bfe...\n",
" searchString: :(q|gh):500<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" meanSea float64 0.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" mp2 (latitude, longitude) float32 nan nan nan ... nan nan\n",
" swh (latitude, longitude) float32 nan nan nan ... nan nan\n",
" mwd (latitude, longitude) float32 nan nan nan ... nan nan\n",
" pp1d (latitude, longitude) float32 nan nan nan ... nan nan\n",
" mwp (latitude, longitude) float32 nan nan nan ... nan nan\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: wave\n",
" description: ECMWF open data\n",
" remote_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" searchString: None