{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "ae136e42", "metadata": {}, "source": [ "# ECMWF Open Data\n", "\n", "This tutorial demonstrates how to use the ECMWF Real-Time Open Data. This data is freely available from ECMWF in GRIB2 format ([👀 Read more ](https://confluence.ecmwf.int/display/DAC/ECMWF+open+data%3A+real-time+forecasts)).\n", "\n", "> \" The data that are becoming available are based on a range of high-resolution forecasts (HRES – 9 km horizontal resolution) and ensemble forecasts (ENS – 18 km horizontal resolution). **They will be made accessible at a resolution of 0.4 x 0.4 degrees**\". \n", "> \\- [Media Statement](https://www.ecmwf.int/en/about/media-centre/news/2022/ecmwf-makes-wide-range-data-openly-available)\n", "\n", "\n", "\n", "**Data Sources**\n", "|`prioriy=`|Data source| Archive Duration |\n", "|--|--|--|\n", "|`\"ecmwf\"` | [ECMWF Open Data](https://data.ecmwf.int/forecasts/) | last 4 days\n", "|`\"azure\"` | [Microsoft Azure](https://ai4edataeuwest.blob.core.windows.net/ecmwf) | 2022-01-21 to present\n", "|`\"aws\"` | [Amazon Web Services](https://ecmwf-forecasts.s3.eu-central-1.amazonaws.com)| 2023-01-18 to present\n", "\n", "**Products**\n", "\n", "|`product=`| Product Description | Available model runs\n", "|--|--|--|\n", "|`\"oper\"` | operational high-resolution forecast, atmospheric fields | `00z`, `12z`, \n", "|`\"wave\"` | wave forecasts | `00z`, `12z`, \n", "|`\"scda\"` | short cut-off high-resolution forecast, atmospheric fields (also known a high-frequency products)\",| `06z`, `18z`\n", "|`\"scwv\"` | short cut-off high-resolution forecast, ocean wave fields (also known a high-frequency products)\",| `06z`, `18z`\n", "|`\"enfo\"` | ensemble forecast, atmospheric fields | `00z`, `06z`, `12z`, `18z`\n", "|`\"waef\"` | ensemble forecast, ocean wave fields, | `00z`, `06z`, `12z`, `18z`\n", "|`\"mmsf\"` | multi-model seasonal forecasts fields from the ECMWF model only. | ?\n", "\n", "**Model Initialization**\n", "\n", "00z, 06z, 12z, 18z. Not all products are available every hour.\n", "\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "b51a1150", "metadata": {}, "outputs": [], "source": [ "from herbie import Herbie\n", "\n", "import cartopy.crs as ccrs\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "from paint.standard2 import cm_tmp, cm_wind, cm_wave_height\n", "from toolbox import EasyMap, pc" ] }, { "cell_type": "code", "execution_count": 2, "id": "b6a467e6", "metadata": { "notebookRunGroups": { "groupValue": "" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Found ┊ model=ecmwf ┊ \u001b[3mproduct=oper\u001b[0m ┊ \u001b[38;2;41;130;13m2023-Feb-01 00:00 UTC\u001b[92m F12\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mGRIB2 @ azure\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mIDX @ azure\u001b[0m\n" ] } ], "source": [ "H = Herbie(\"2023-02-01\", model=\"ecmwf\", product=\"oper\", fxx=12)" ] }, { "cell_type": "code", "execution_count": 4, "id": "b77872fe", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Found ┊ model=ecmwf ┊ \u001b[3mproduct=oper\u001b[0m ┊ \u001b[38;2;41;130;13m2023-Feb-01 06:00 UTC\u001b[92m F12\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mGRIB2 @ azure-scda\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mIDX @ azure-scda\u001b[0m\n" ] }, { "data": { "text/plain": [ "'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20230201/06z/0p4-beta/scda/20230201060000-12h-scda-fc.grib2'" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "H = Herbie(\"2023-02-01 06\", model=\"ecmwf\", product=\"oper\", fxx=12)\n", "H.grib" ] }, { "cell_type": "code", "execution_count": 20, "id": "18a85e40", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ Found ┊ model=ecmwf ┊ \u001b[3mproduct=oper\u001b[0m ┊ \u001b[38;2;41;130;13m2023-Jan-23 00:00 UTC\u001b[92m F12\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mGRIB2 @ azure\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3mIDX @ azure\u001b[0m\n" ] } ], "source": [ "H = Herbie(\"2023-1-23\", model=\"ecmwf\", product=\"oper\", fxx=12, priority=\"azure\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "e9acd907", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "👨🏻🏭 Created directory: [/home/blaylock/data/ecmwf/20230201]\n", "curl -s --range 16444658-17053704 \"https://ai4edataeuwest.blob.core.windows.net/ecmwf/20230201/00z/0p4-beta/oper/20230201000000-12h-oper-fc.grib2\" > \"/home/blaylock/data/ecmwf/20230201/subset_d712f340__20230201000000-12h-oper-fc.grib2\"\n" ] }, { "data": { "text/html": [ "
<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2023-02-01\n",
" step timedelta64[ns] 12:00:00\n",
" heightAboveGround float64 2.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2023-02-01T12:00:00\n",
"Data variables:\n",
" t2m (latitude, longitude) float32 246.4 246.4 ... 240.2\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...\n",
" local_grib: /home/blaylock/data/ecmwf/20230201/subset_d712f3...\n",
" searchString: :2t:| \n", " | grib_message | \n", "start_byte | \n", "end_byte | \n", "range | \n", "reference_time | \n", "valid_time | \n", "step | \n", "param | \n", "levelist | \n", "levtype | \n", "number | \n", "domain | \n", "expver | \n", "class | \n", "type | \n", "stream | \n", "search_this | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "1 | \n", "0 | \n", "609046 | \n", "0-609046 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "skt | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":skt:sfc:g:0001:od:fc:oper | \n", "
| 1 | \n", "2 | \n", "609046 | \n", "1218092 | \n", "609046-1218092 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "st | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":st:sfc:g:0001:od:fc:oper | \n", "
| 2 | \n", "3 | \n", "1218092 | \n", "1827138 | \n", "1218092-1827138 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "gh | \n", "1000 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":gh:1000:pl:g:0001:od:fc:oper | \n", "
| 3 | \n", "4 | \n", "1827138 | \n", "2436184 | \n", "1827138-2436184 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "gh | \n", "925 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":gh:925:pl:g:0001:od:fc:oper | \n", "
| 4 | \n", "5 | \n", "2436184 | \n", "2842280 | \n", "2436184-2842280 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "r | \n", "300 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":r:300:pl:g:0001:od:fc:oper | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 77 | \n", "78 | \n", "48114266 | \n", "48926262 | \n", "48114266-48926262 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "d | \n", "250 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":d:250:pl:g:0001:od:fc:oper | \n", "
| 78 | \n", "79 | \n", "48926262 | \n", "49738258 | \n", "48926262-49738258 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "vo | \n", "250 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":vo:250:pl:g:0001:od:fc:oper | \n", "
| 79 | \n", "80 | \n", "49738258 | \n", "50550254 | \n", "49738258-50550254 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "vo | \n", "50 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":vo:50:pl:g:0001:od:fc:oper | \n", "
| 80 | \n", "81 | \n", "50550254 | \n", "51159324 | \n", "50550254-51159324 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "ro | \n", "NaN | \n", "sfc | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":ro:sfc:g:0001:od:fc:oper | \n", "
| 81 | \n", "82 | \n", "51159324 | \n", "51971320 | \n", "51159324-51971320 | \n", "2022-01-26 | \n", "2022-01-26 12:00:00 | \n", "0 days 12:00:00 | \n", "d | \n", "50 | \n", "pl | \n", "NaN | \n", "g | \n", "0001 | \n", "od | \n", "fc | \n", "oper | \n", ":d:50:pl:g:0001:od:fc:oper | \n", "
82 rows × 17 columns
\n", "<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 12:00:00\n",
" heightAboveGround float64 2.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] ...\n",
"Data variables:\n",
" t2m (latitude, longitude) float32 ...\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bf1...\n",
" searchString: :2t:<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" heightAboveGround float64 10.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" u10 (latitude, longitude) float32 8.174 8.174 ... -5.998\n",
" v10 (latitude, longitude) float32 3.499 3.499 ... 3.733\n",
" gribfile_projection object None\n",
" spd (latitude, longitude) float32 8.891 8.891 ... 7.065\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bfe...\n",
" searchString: :10(u|v):<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" isobaricInhPa float64 500.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" q (latitude, longitude) float32 6.679e-05 ... 0.0001641\n",
" gh (latitude, longitude) float32 5.029e+03 ... 5.136e+03\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: oper\n",
" description: ECMWF open data\n",
" remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\subset_bfe...\n",
" searchString: :(q|gh):500<xarray.Dataset>\n",
"Dimensions: (latitude: 451, longitude: 900)\n",
"Coordinates:\n",
" time datetime64[ns] 2022-01-26\n",
" step timedelta64[ns] 00:00:00\n",
" meanSea float64 0.0\n",
" * latitude (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0\n",
" * longitude (longitude) float64 -180.0 -179.6 ... 179.2 179.6\n",
" valid_time datetime64[ns] 2022-01-26\n",
"Data variables:\n",
" mp2 (latitude, longitude) float32 nan nan nan ... nan nan\n",
" swh (latitude, longitude) float32 nan nan nan ... nan nan\n",
" mwd (latitude, longitude) float32 nan nan nan ... nan nan\n",
" pp1d (latitude, longitude) float32 nan nan nan ... nan nan\n",
" mwp (latitude, longitude) float32 nan nan nan ... nan nan\n",
" gribfile_projection object None\n",
"Attributes:\n",
" GRIB_edition: 2\n",
" GRIB_centre: ecmf\n",
" GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts\n",
" GRIB_subCentre: 0\n",
" Conventions: CF-1.7\n",
" institution: European Centre for Medium-Range Weather Forecasts\n",
" model: ecmwf\n",
" product: wave\n",
" description: ECMWF open data\n",
" remote_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" local_grib: C:\\Users\\blaylock\\data\\ecmwf\\20220126\\2022012600...\n",
" searchString: None