ECMWF (IFS and AIFS)#
This tutorial demonstrates how to download the ECMWF Open Data Integrated Forecast System (IFS) and Artificial Intelligence IFS (AIFS).
Recent and archived IFS and AIFS data is freely available from ECMWF in GRIB2 format (π Read more).
βThe data that are becoming available are based on a range of high-resolution forecasts (HRES β 9 km horizontal resolution) and ensemble forecasts (ENS β 18 km horizontal resolution).
29 February 2024: Update from 0.4 degree resolution to 0.25 degree resolution
Data Availability#
Jan 18, 2023, IFS 0.4 degree resolution first available
Feb 1, 2024 IFS 0.25 degree resolution first available
Feb 1, 2024 AIFS 0.25 degree resolution first available
May ?, 2024 IFS 0.4 degree discontinued.
Model Types#
ECMWF provides data for two different models
model="ifs"
ECMWF Integrated Forecast Systemmodel="aifs"
ECMWF Artificial Intelligence Integrated Forecast System
Data Source#
|
Data source |
Archive Duration |
---|---|---|
|
last 4 days |
|
|
2022-01-21 to present |
|
|
2023-01-18 to present |
Products#
Note: the
aifs
only has theoper
product.
|
Product Description |
Available model runs |
---|---|---|
|
operational high-resolution forecast, atmospheric fields |
|
|
wave forecasts |
|
|
short cut-off high-resolution forecast, atmospheric fields (also known a high-frequency products)β, |
|
|
short cut-off high-resolution forecast, ocean wave fields (also known a high-frequency products)β, |
|
|
ensemble forecast, atmospheric fields |
|
|
ensemble forecast, ocean wave fields, |
|
|
multi-model seasonal forecasts fields from the ECMWF model only. |
? |
Model initialized at 00z, 06z, 12z, 18z, but not all products are available every hour.
[1]:
from herbie import Herbie
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import numpy as np
from herbie import paint
from herbie.toolbox import EasyMap, pc
Integrated Forecast System (IFS)#
IFS data is only available at 0.4 degree prior to February 1, 2024. After that date, the IFS is available at 0.25 degree resolution.
[2]:
H = Herbie("2024-03-1", model="ifs", product="oper", fxx=12)
H.grib, H.idx
β
Found β model=ifs β product=oper β 2024-Mar-01 00:00 UTC F12 β GRIB2 @ azure β IDX @ azure
[2]:
('https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2',
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.index')
[3]:
# Show the inventory
H.inventory()
[3]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 1 | 0 | 798588 | 0-798588 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | tp | NaN | sfc | NaN | g | 0001 | od | fc | oper | :tp:sfc:g:0001:od:fc:oper |
1 | 2 | 798588 | 1327999 | 798588-1327999 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | r | 500 | pl | NaN | g | 0001 | od | fc | oper | :r:500:pl:g:0001:od:fc:oper |
2 | 3 | 1327999 | 1817293 | 1327999-1817293 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | gh | 850 | pl | NaN | g | 0001 | od | fc | oper | :gh:850:pl:g:0001:od:fc:oper |
3 | 4 | 1817293 | 2534675 | 1817293-2534675 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | u | 925 | pl | NaN | g | 0001 | od | fc | oper | :u:925:pl:g:0001:od:fc:oper |
4 | 5 | 2534675 | 3268614 | 2534675-3268614 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | v | 925 | pl | NaN | g | 0001 | od | fc | oper | :v:925:pl:g:0001:od:fc:oper |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
78 | 79 | 58285210 | 59656192 | 58285210-59656192 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | d | 250 | pl | NaN | g | 0001 | od | fc | oper | :d:250:pl:g:0001:od:fc:oper |
79 | 80 | 59656192 | 59759143 | 59656192-59759143 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | ro | NaN | sfc | NaN | g | 0001 | od | fc | oper | :ro:sfc:g:0001:od:fc:oper |
80 | 81 | 59759143 | 61078909 | 59759143-61078909 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | vo | 250 | pl | NaN | g | 0001 | od | fc | oper | :vo:250:pl:g:0001:od:fc:oper |
81 | 82 | 61078909 | 62399470 | 61078909-62399470 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | d | 50 | pl | NaN | g | 0001 | od | fc | oper | :d:50:pl:g:0001:od:fc:oper |
82 | 83 | 62399470 | 63520541 | 62399470-63520541 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | vo | 50 | pl | NaN | g | 0001 | od | fc | oper | :vo:50:pl:g:0001:od:fc:oper |
83 rows Γ 17 columns
[5]:
# Show just 10-m U and V wind
H.inventory(":10[u|v]:")
[5]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
46 | 47 | 28201794 | 29068335 | 28201794-29068335 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | 10u | NaN | sfc | NaN | g | 0001 | od | fc | oper | :10u:sfc:g:0001:od:fc:oper |
47 | 48 | 29068335 | 29928681 | 29068335-29928681 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | 10v | NaN | sfc | NaN | g | 0001 | od | fc | oper | :10v:sfc:g:0001:od:fc:oper |
[6]:
# Get 2-m temperature as an xarray Dataset
ds = H.xarray(":2t:", verbose=True)
ds
π¨π»βπ Created directory: [/home/blaylock/data/ifs/20240301]
π Download subset: ββHerbie IFS model oper product initialized 2024-Mar-01 00:00 UTC F12 β source=azure
cURL from https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2
Found 1 grib messages.
Download subset group 1
41 :2t:sfc:g:0001:od:fc:oper
curl -s --range 24427114-25090915 "https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2" > "/home/blaylock/data/ifs/20240301/subset_e0127a9f__20240301000000-12h-oper-fc.grib2"
πΎ Saved the subset to /home/blaylock/data/ifs/20240301/subset_e0127a9f__20240301000000-12h-oper-fc.grib2
[6]:
<xarray.Dataset> Size: 4MB Dimensions: (latitude: 721, longitude: 1440) Coordinates: time datetime64[ns] 8B 2024-03-01 step timedelta64[ns] 8B 12:00:00 heightAboveGround float64 8B 2.0 * latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0 * longitude (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8 valid_time datetime64[ns] 8B 2024-03-01T12:00:00 gribfile_projection object 8B None Data variables: t2m (latitude, longitude) float32 4MB 244.9 244.9 ... 224.6 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: oper description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20240301/subset_e0127a9f... search: :2t:
[7]:
ds.t2m.plot()
[7]:
<matplotlib.collections.QuadMesh at 0x7fb7ef104da0>
0.4 degree IFS#
Again, the 0.4 degree IFS was available starting January 2023, but will be discontinued in May 2024 in favor of the 0.25 degree data. You can still access these files as long as they exist.
[8]:
# Accessing historical
H = Herbie("2023-07-04", model="ifs", product="oper", fxx=12)
H.grib
β
Found β model=ifs β product=oper β 2023-Jul-04 00:00 UTC F12 β GRIB2 @ azure β IDX @ azure
[8]:
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20230704/00z/0p4-beta/oper/20230704000000-12h-oper-fc.grib2'
Eccodes-style Index Files#
The ECMWF index files are different than the wgrib2-style index files, so pay close attention to how you should select the field you want.
[9]:
H = Herbie("2024-03-1", model="ifs", product="oper", fxx=12)
# Show the search_help
print(H.search_help)
β
Found β model=ifs β product=oper β 2024-Mar-01 00:00 UTC F12 β GRIB2 @ azure β IDX @ azure
Use regular expression to search for lines in the index file.
Here are some examples you can use for the ecCodes-style `search`
Look at the ECMWF GRIB Parameter Database
https://apps.ecmwf.int/codes/grib/param-db
product=`oper` or `enfo`
======================== ==============================================
search= GRIB messages that will be downloaded
======================== ==============================================
":2t:" 2-m temperature
":2d:" 2-m dew point temperature
":10u:" 10-m u wind vector
":10v:" 10-m v wind vector
":10[uv]: 10-m u and 10-m v wind
":[tuvr]:" Temp, u/v wind, RH (all levels)
":500:" All variables on the 500 hPa level
":gh:500" Geopotential height only at 500 hPa
":gh:" Geopotential height (all pressure levels)
":t:" Temperature (all pressure levels)
":q:" Specific Humidity (all pressure levels)
":r:" Relative humidity (all pressure levels)
":v:" v wind vector (all pressure levels)
":u:" u wind vector (all pressure levels)
":w:" Vertical velocity (Pascals per second)
":lsm:" Land-sea mask
":ttr:" Top net long-wave (thermal) radiation
":ssrd:" Surface short-wave (solar) radiation downwards
":ssr:" Surface net short-wave (solar) radiation
":strd:" Surface long-wave (thermal) radiation downwards
":str:" Surface net long-wave (thermal) radiation
":swvl1:" Volumetric soil water layer 1 (depth 0 meters)
":swvl2:" Volumetric soil water layer 2 (depth 7 meters)
":swvl3:" Volumetric soil water layer 3 (depth 28 meters)
":swvl4:" Volumetric soil water layer 4 (depth 100 meters)
":skt:" Skin (surface) temperature
":d:" Divergence (all levels)
":st:" Soil temperature
":stl2:" Soil temperature level 2 (depth 7 meters)
":tp:" Total precipitation
":ro:" Run-off
":asn:" Snow albedo
":msl:" Mean sea level pressure
":sp:" Surface pressure
":cape:" CAPE
":tcwv:" Total column vertically integrated water vapor
":vo:" Relative vorticity
product=`wave` or `waef`
======================== ==============================================
search= GRIB messages that will be downloaded
======================== ==============================================
":swh:" Significant height of wind waves + swell
":mwp:" Mean wave period
":mwd:" Mean wave direction
":pp1d:" Peak wave period
":mp2:" Mean zero-crossing wave period
If you need help with regular expression, search the web or look at
this cheatsheet: https://www.petefreitag.com/cheatsheets/regex/.
Here is an example: https://regex101.com/r/niNjwp/1
When considering search queries, pay attention to the βsearch_thisβ column; that columns is used for the regex search.
[10]:
H.inventory()
[10]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 1 | 0 | 798588 | 0-798588 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | tp | NaN | sfc | NaN | g | 0001 | od | fc | oper | :tp:sfc:g:0001:od:fc:oper |
1 | 2 | 798588 | 1327999 | 798588-1327999 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | r | 500 | pl | NaN | g | 0001 | od | fc | oper | :r:500:pl:g:0001:od:fc:oper |
2 | 3 | 1327999 | 1817293 | 1327999-1817293 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | gh | 850 | pl | NaN | g | 0001 | od | fc | oper | :gh:850:pl:g:0001:od:fc:oper |
3 | 4 | 1817293 | 2534675 | 1817293-2534675 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | u | 925 | pl | NaN | g | 0001 | od | fc | oper | :u:925:pl:g:0001:od:fc:oper |
4 | 5 | 2534675 | 3268614 | 2534675-3268614 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | v | 925 | pl | NaN | g | 0001 | od | fc | oper | :v:925:pl:g:0001:od:fc:oper |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
78 | 79 | 58285210 | 59656192 | 58285210-59656192 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | d | 250 | pl | NaN | g | 0001 | od | fc | oper | :d:250:pl:g:0001:od:fc:oper |
79 | 80 | 59656192 | 59759143 | 59656192-59759143 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | ro | NaN | sfc | NaN | g | 0001 | od | fc | oper | :ro:sfc:g:0001:od:fc:oper |
80 | 81 | 59759143 | 61078909 | 59759143-61078909 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | vo | 250 | pl | NaN | g | 0001 | od | fc | oper | :vo:250:pl:g:0001:od:fc:oper |
81 | 82 | 61078909 | 62399470 | 61078909-62399470 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | d | 50 | pl | NaN | g | 0001 | od | fc | oper | :d:50:pl:g:0001:od:fc:oper |
82 | 83 | 62399470 | 63520541 | 62399470-63520541 | 2024-03-01 | 2024-03-01 12:00:00 | 0 days 12:00:00 | vo | 50 | pl | NaN | g | 0001 | od | fc | oper | :vo:50:pl:g:0001:od:fc:oper |
83 rows Γ 17 columns
[18]:
H.inventory().param.unique()
[18]:
array(['tp', 'msl', '2t', '10u', 'gh', 't', 'u', 'v', 'st', 'r', 'skt',
'sp', 'lsm', '10v', 'tcwv', 'q', 'd', 'vo', 'ro'], dtype=object)
Ok, now that we have some understanding of the index file, we can read the 2-m temperature data.
[11]:
ds = H.xarray(":2t:")
ds
[11]:
<xarray.Dataset> Size: 4MB Dimensions: (latitude: 721, longitude: 1440) Coordinates: time datetime64[ns] 8B 2024-03-01 step timedelta64[ns] 8B 12:00:00 heightAboveGround float64 8B 2.0 * latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0 * longitude (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8 valid_time datetime64[ns] 8B 2024-03-01T12:00:00 gribfile_projection object 8B None Data variables: t2m (latitude, longitude) float32 4MB 244.9 244.9 ... 224.6 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: oper description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20240301/subset_e0127a9f... search: :2t:
[12]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
ds.longitude,
ds.latitude,
ds.t2m - 273.15,
transform=pc,
**paint.NWSTemperature.kwargs2,
)
plt.colorbar(
p, ax=ax, orientation="horizontal", pad=0.05, **paint.NWSTemperature.cbar_kwargs2
)
ax.set_title(
f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
loc="left",
)
ax.set_title(ds.t2m.GRIB_name, loc="right")
[12]:
Text(1.0, 1.0, '2 metre temperature')
Now the same, but for wind. Use the Herbie accessor to compute wind speed and direction.
[14]:
H = Herbie("2024-03-01", model="ifs", product="oper")
# Get u and v wind component
ds = H.xarray(":10[u|v]:").herbie.with_wind()
ds
β
Found β model=ifs β product=oper β 2024-Mar-01 00:00 UTC F00 β GRIB2 @ azure β IDX @ azure
[14]:
<xarray.Dataset> Size: 17MB Dimensions: (latitude: 721, longitude: 1440) Coordinates: time datetime64[ns] 8B 2024-03-01 step timedelta64[ns] 8B 00:00:00 heightAboveGround float64 8B 10.0 * latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0 * longitude (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8 valid_time datetime64[ns] 8B 2024-03-01 gribfile_projection object 8B None Data variables: u10 (latitude, longitude) float32 4MB 1.654 ... -1.237 v10 (latitude, longitude) float32 4MB 0.9546 ... 2.361 si10 (latitude, longitude) float32 4MB 1.909 1.909 ... 2.665 wdir10 (latitude, longitude) float32 4MB 240.0 240.0 ... 152.3 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: oper description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20240301/subset_e0ef1f8f... search: :10[u|v]:
[16]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
ds.longitude, ds.latitude, ds.si10, transform=pc, **paint.NWSWindSpeed.kwargs2
)
plt.colorbar(
p, ax=ax, orientation="horizontal", pad=0.05, **paint.NWSWindSpeed.cbar_kwargs2
)
ax.set_title(
f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
loc="left",
)
ax.set_title(ds.si10.standard_name, loc="right")
[16]:
Text(1.0, 1.0, 'wind_speed')
Now lets get the humidity and geopotential height at 500 hPa
[17]:
ds = H.xarray(":(?:q|gh):500")
ds
[17]:
<xarray.Dataset> Size: 8MB Dimensions: (latitude: 721, longitude: 1440) Coordinates: time datetime64[ns] 8B 2024-03-01 step timedelta64[ns] 8B 00:00:00 isobaricInhPa float64 8B 500.0 * latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0 * longitude (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8 valid_time datetime64[ns] 8B 2024-03-01 gribfile_projection object 8B None Data variables: q (latitude, longitude) float32 4MB 0.0001736 ... 0.00... gh (latitude, longitude) float32 4MB 5.196e+03 ... 4.89... Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: oper description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20240301/subset_e0efbf07... search: :(?:q|gh):500
[19]:
# NOTE: transforming data to a Robinson projection can take some time...
ax = EasyMap("50m", crs=ccrs.Robinson(), figsize=[10, 10]).STATES().BORDERS().ax
# Color shade by specific humidity
p = ax.pcolormesh(ds.longitude, ds.latitude, ds.q, transform=pc, cmap="Greens")
plt.colorbar(
p,
ax=ax,
orientation="horizontal",
pad=0.05,
label=f"{ds.q.GRIB_name} ({ds.q.units})",
)
# Contours for geopotential height
ax.contour(
ds.longitude,
ds.latitude,
ds.gh,
transform=pc,
colors="k",
linewidths=0.5,
levels=range(0, 10_000, 60 * 2),
)
ax.set_title(
f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
loc="left",
)
ax.set_title(
f"{ds.isobaricInhPa.item()} {ds.isobaricInhPa.units}\n{ds.q.GRIB_name}/{ds.gh.GRIB_name}",
loc="right",
)
[19]:
Text(1.0, 1.0, '500.0 hPa\nSpecific humidity/Geopotential height')
The Kernel crashed while executing code in the current cell or a previous cell.
Please review the code in the cell(s) to identify a possible cause of the failure.
Click <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info.
View Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details.
ECMWF IFS Wave Output#
[2]:
H = Herbie("2022-01-26 00:00", model="ifs", product="wave")
β
Found β model=ifs β product=wave β 2022-Jan-26 00:00 UTC F00 β GRIB2 @ azure β IDX @ azure
[3]:
ds = H.xarray(None, verbose=True)
ds
/home/blaylock/GITHUB/Herbie/herbie/core.py:1102: UserWarning: Will not remove GRIB file because Herbie will only remove subsetted files (not full files).
warnings.warn(
β
Success! Downloaded IFS from azure
src: https://ai4edataeuwest.blob.core.windows.net/ecmwf/20220126/00z/0p4-beta/wave/20220126000000-0h-wave-fc.grib2
dst: /home/blaylock/data/ifs/20220126/20220126000000-0h-wave-fc.grib2
[3]:
<xarray.Dataset> Size: 8MB Dimensions: (latitude: 451, longitude: 900) Coordinates: time datetime64[ns] 8B 2022-01-26 step timedelta64[ns] 8B 00:00:00 meanSea float64 8B 0.0 * latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0 * longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6 valid_time datetime64[ns] 8B 2022-01-26 gribfile_projection object 8B None Data variables: mp2 (latitude, longitude) float32 2MB ... swh (latitude, longitude) float32 2MB ... mwd (latitude, longitude) float32 2MB ... pp1d (latitude, longitude) float32 2MB ... mwp (latitude, longitude) float32 2MB ... Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: wave description: ECMWF Open Data - Integrated Forecast System remote_grib: /home/blaylock/data/ifs/20220126/20220126000000-... local_grib: /home/blaylock/data/ifs/20220126/20220126000000-... search: None
[4]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
ds.longitude,
ds.latitude,
ds.swh,
transform=pc,
**paint.NWSWaveHeight.kwargs2,
)
plt.colorbar(
p,
ax=ax,
orientation="horizontal",
pad=0.05,
**paint.NWSWaveHeight.cbar_kwargs2,
)
ax.set_title(
f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
loc="left",
)
ax.set_title(ds.swh.GRIB_name, loc="right")
[4]:
Text(1.0, 1.0, 'Significant height of combined wind waves and swell')
Ensemble Forecast Products#
[5]:
H = Herbie("2022-01-26 00:00", model="ifs", product="enfo")
ds = H.xarray(":2t:")
ds
β
Found β model=ifs β product=enfo β 2022-Jan-26 00:00 UTC F00 β GRIB2 @ azure β IDX @ azure
Note: Returning a list of [2] xarray.Datasets because cfgrib opened with multiple hypercubes.
[5]:
[<xarray.Dataset> Size: 81MB
Dimensions: (number: 50, latitude: 451, longitude: 900)
Coordinates:
* number (number) int64 400B 1 2 3 4 5 6 7 ... 45 46 47 48 49 50
time datetime64[ns] 8B 2022-01-26
step timedelta64[ns] 8B 00:00:00
heightAboveGround float64 8B 2.0
* latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
* longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
valid_time datetime64[ns] 8B 2022-01-26
gribfile_projection object 8B None
Data variables:
t2m (number, latitude, longitude) float32 81MB 246.5 ......
Attributes:
GRIB_edition: 2
GRIB_centre: ecmf
GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts
GRIB_subCentre: 0
Conventions: CF-1.7
institution: European Centre for Medium-Range Weather Forecasts
model: ifs
product: enfo
description: ECMWF Open Data - Integrated Forecast System
remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...
local_grib: /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
search: :2t:,
<xarray.Dataset> Size: 2MB
Dimensions: (latitude: 451, longitude: 900)
Coordinates:
number int64 8B 0
time datetime64[ns] 8B 2022-01-26
step timedelta64[ns] 8B 00:00:00
heightAboveGround float64 8B 2.0
* latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
* longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
valid_time datetime64[ns] 8B 2022-01-26
gribfile_projection object 8B None
Data variables:
t2m (latitude, longitude) float32 2MB 246.2 246.2 ... 246.3
Attributes:
GRIB_edition: 2
GRIB_centre: ecmf
GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts
GRIB_subCentre: 0
Conventions: CF-1.7
institution: European Centre for Medium-Range Weather Forecasts
model: ifs
product: enfo
description: ECMWF Open Data - Integrated Forecast System
remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm...
local_grib: /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
search: :2t:]
[6]:
# Dataset with all 50 members
ds[0]
[6]:
<xarray.Dataset> Size: 81MB Dimensions: (number: 50, latitude: 451, longitude: 900) Coordinates: * number (number) int64 400B 1 2 3 4 5 6 7 ... 45 46 47 48 49 50 time datetime64[ns] 8B 2022-01-26 step timedelta64[ns] 8B 00:00:00 heightAboveGround float64 8B 2.0 * latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0 * longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6 valid_time datetime64[ns] 8B 2022-01-26 gribfile_projection object 8B None Data variables: t2m (number, latitude, longitude) float32 81MB 246.5 ...... Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: enfo description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20220126/subset_bfef7f9f... search: :2t:
[7]:
# This Dataset is of the mean of all the members, right?
ds[1]
[7]:
<xarray.Dataset> Size: 2MB Dimensions: (latitude: 451, longitude: 900) Coordinates: number int64 8B 0 time datetime64[ns] 8B 2022-01-26 step timedelta64[ns] 8B 00:00:00 heightAboveGround float64 8B 2.0 * latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0 * longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6 valid_time datetime64[ns] 8B 2022-01-26 gribfile_projection object 8B None Data variables: t2m (latitude, longitude) float32 2MB 246.2 246.2 ... 246.3 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: enfo description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20220126/subset_bfef7f9f... search: :2t:
[8]:
H.idx
[8]:
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20220126/00z/0p4-beta/enfo/20220126000000-0h-enfo-ef.index'
Ensemble Wave Products#
[16]:
H = Herbie("2022-01-26 00:00", model="ifs", product="waef")
H.inventory()
β
Found β model=ifs β product=waef β 2022-Jan-26 00:00 UTC F00 β GRIB2 @ azure β IDX @ azure
[16]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 1 | 0 | 501693 | 0-501693 | 2022-01-26 | 2022-01-26 | 0 days | swh | NaN | sfc | 38 | g | 0001 | od | pf | waef | :swh:sfc:38:g:0001:od:pf:waef |
1 | 2 | 501693 | 1003064 | 501693-1003064 | 2022-01-26 | 2022-01-26 | 0 days | swh | NaN | sfc | 46 | g | 0001 | od | pf | waef | :swh:sfc:46:g:0001:od:pf:waef |
2 | 3 | 1003064 | 1504757 | 1003064-1504757 | 2022-01-26 | 2022-01-26 | 0 days | swh | NaN | sfc | 43 | g | 0001 | od | pf | waef | :swh:sfc:43:g:0001:od:pf:waef |
3 | 4 | 1504757 | 2006096 | 1504757-2006096 | 2022-01-26 | 2022-01-26 | 0 days | swh | NaN | sfc | 40 | g | 0001 | od | pf | waef | :swh:sfc:40:g:0001:od:pf:waef |
4 | 5 | 2006096 | 2507435 | 2006096-2507435 | 2022-01-26 | 2022-01-26 | 0 days | swh | NaN | sfc | NaN | g | 0001 | od | cf | waef | :swh:sfc:g:0001:od:cf:waef |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
250 | 251 | 125340988 | 125842349 | 125340988-125842349 | 2022-01-26 | 2022-01-26 | 0 days | pp1d | NaN | sfc | 7 | g | 0001 | od | pf | waef | :pp1d:sfc:7:g:0001:od:pf:waef |
251 | 252 | 125842349 | 126343438 | 125842349-126343438 | 2022-01-26 | 2022-01-26 | 0 days | pp1d | NaN | sfc | 29 | g | 0001 | od | pf | waef | :pp1d:sfc:29:g:0001:od:pf:waef |
252 | 253 | 126343438 | 126844527 | 126343438-126844527 | 2022-01-26 | 2022-01-26 | 0 days | pp1d | NaN | sfc | 34 | g | 0001 | od | pf | waef | :pp1d:sfc:34:g:0001:od:pf:waef |
253 | 254 | 126844527 | 127345908 | 126844527-127345908 | 2022-01-26 | 2022-01-26 | 0 days | pp1d | NaN | sfc | NaN | g | 0001 | od | cf | waef | :pp1d:sfc:g:0001:od:cf:waef |
254 | 255 | 127345908 | 127847247 | 127345908-127847247 | 2022-01-26 | 2022-01-26 | 0 days | mp2 | NaN | sfc | NaN | g | 0001 | od | cf | waef | :mp2:sfc:g:0001:od:cf:waef |
255 rows Γ 17 columns
[18]:
H = Herbie("2022-01-26", model="ifs", product="enfo")
H.inventory()
β
Found β model=ifs β product=enfo β 2022-Jan-26 00:00 UTC F00 β GRIB2 @ azure β IDX @ azure
[18]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 1 | 0 | 609069 | 0-609069 | 2022-01-26 | 2022-01-26 | 0 days | 2t | NaN | sfc | 29 | g | 0001 | od | pf | enfo | :2t:sfc:29:g:0001:od:pf:enfo |
1 | 2 | 609069 | 1218138 | 609069-1218138 | 2022-01-26 | 2022-01-26 | 0 days | 10u | NaN | sfc | 27 | g | 0001 | od | pf | enfo | :10u:sfc:27:g:0001:od:pf:enfo |
2 | 3 | 1218138 | 1827207 | 1218138-1827207 | 2022-01-26 | 2022-01-26 | 0 days | 10v | NaN | sfc | 14 | g | 0001 | od | pf | enfo | :10v:sfc:14:g:0001:od:pf:enfo |
3 | 4 | 1827207 | 2436276 | 1827207-2436276 | 2022-01-26 | 2022-01-26 | 0 days | 2t | NaN | sfc | 2 | g | 0001 | od | pf | enfo | :2t:sfc:2:g:0001:od:pf:enfo |
4 | 5 | 2436276 | 3045345 | 2436276-3045345 | 2022-01-26 | 2022-01-26 | 0 days | 10u | NaN | sfc | 43 | g | 0001 | od | pf | enfo | :10u:sfc:43:g:0001:od:pf:enfo |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
4177 | 4178 | 2574120261 | 2574932280 | 2574120261-2574932280 | 2022-01-26 | 2022-01-26 | 0 days | d | 200 | pl | 20 | g | 0001 | od | pf | enfo | :d:200:pl:20:g:0001:od:pf:enfo |
4178 | 4179 | 2574932280 | 2575744299 | 2574932280-2575744299 | 2022-01-26 | 2022-01-26 | 0 days | d | 200 | pl | 41 | g | 0001 | od | pf | enfo | :d:200:pl:41:g:0001:od:pf:enfo |
4179 | 4180 | 2575744299 | 2576556318 | 2575744299-2576556318 | 2022-01-26 | 2022-01-26 | 0 days | d | 200 | pl | 4 | g | 0001 | od | pf | enfo | :d:200:pl:4:g:0001:od:pf:enfo |
4180 | 4181 | 2576556318 | 2577368337 | 2576556318-2577368337 | 2022-01-26 | 2022-01-26 | 0 days | d | 200 | pl | 50 | g | 0001 | od | pf | enfo | :d:200:pl:50:g:0001:od:pf:enfo |
4181 | 4182 | 2577368337 | 2578180356 | 2577368337-2578180356 | 2022-01-26 | 2022-01-26 | 0 days | d | 200 | pl | 23 | g | 0001 | od | pf | enfo | :d:200:pl:23:g:0001:od:pf:enfo |
4182 rows Γ 17 columns
Here is another examle, just for fun
[21]:
H = Herbie("2022-01-26", model="ifs", product="oper", fxx=12)
H
β
Found β model=ifs β product=oper β 2022-Jan-26 00:00 UTC F12 β GRIB2 @ azure β IDX @ azure
[21]:
ββHerbie IFS model oper product initialized 2022-Jan-26 00:00 UTC F12 β source=azure
[22]:
# Download just the 10-m u and v winds
ds = H.download(search=":10(u|v):")
ds
/home/blaylock/GITHUB/Herbie/herbie/core.py:808: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract.
logic = df.search_this.str.contains(search)
[22]:
PosixPath('/home/blaylock/data/ifs/20220126/subset_bf121a77__20220126000000-12h-oper-fc.grib2')
[23]:
# Retrieve the 500 hPa temperature as an xarray.Dataset
ds = H.xarray(search=":t:500:")
ds
[23]:
<xarray.Dataset> Size: 2MB Dimensions: (latitude: 451, longitude: 900) Coordinates: time datetime64[ns] 8B 2022-01-26 step timedelta64[ns] 8B 12:00:00 isobaricInhPa float64 8B 500.0 * latitude (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0 * longitude (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6 valid_time datetime64[ns] 8B 2022-01-26T12:00:00 gribfile_projection object 8B None Data variables: t (latitude, longitude) float32 2MB 234.9 234.9 ... 242.5 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: ifs product: oper description: ECMWF Open Data - Integrated Forecast System remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/ifs/20220126/subset_bf124169... search: :t:500:
[24]:
ds.t.plot()
[24]:
<matplotlib.collections.QuadMesh at 0x7f756deb8500>
Artificial Intelligence IFS#
Download and plot ECMWF AIFS model data.
β AIFS GRIB files before March 13, 2024, seemed to be encoded incorrectly as they are read as a vector instead of a grid. Iβm not sure how this data can be recovered or reshaped.
[29]:
H = Herbie("2024-06-11", model="aifs", product="oper", fxx=12)
H.grib, H.idx
β
Found β model=aifs β product=oper β 2024-Jun-11 00:00 UTC F12 β GRIB2 @ azure β IDX @ azure
[29]:
('https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240611/00z/aifs/0p25/oper/20240611000000-12h-oper-fc.grib2',
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240611/00z/aifs/0p25/oper/20240611000000-12h-oper-fc.index')
[30]:
H.inventory(":t:850")
[30]:
grib_message | start_byte | end_byte | range | reference_time | valid_time | step | param | levelist | levtype | number | domain | expver | class | type | stream | search_this | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
32 | 33 | 25515811 | 26027206 | 25515811-26027206 | 2024-06-11 | 2024-06-11 12:00:00 | 0 days 12:00:00 | t | 850 | pl | NaN | g | 0001 | ai | fc | oper | :t:850:pl:g:0001:ai:fc:oper |
[42]:
ds = H.xarray(":t:850")
(ds.t - 273.15).plot(
x="longitude",
y="latitude",
ax=EasyMap().ax,
transform=pc,
**paint.NWSTemperature.kwargs2,
cbar_kwargs=paint.NWSTemperature.cbar_kwargs2
| dict(orientation="horizontal", pad=0.01),
)
ds
[42]:
<xarray.Dataset> Size: 4MB Dimensions: (latitude: 721, longitude: 1440) Coordinates: time datetime64[ns] 8B 2024-06-11 step timedelta64[ns] 8B 12:00:00 isobaricInhPa float64 8B 850.0 * latitude (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0 * longitude (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8 valid_time datetime64[ns] 8B 2024-06-11T12:00:00 gribfile_projection object 8B None Data variables: t (latitude, longitude) float32 4MB 271.1 271.1 ... 235.1 Attributes: GRIB_edition: 2 GRIB_centre: ecmf GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts GRIB_subCentre: 0 Conventions: CF-1.7 institution: European Centre for Medium-Range Weather Forecasts model: aifs product: oper description: ECMWF Open Data - Artificial Inteligence Integra... remote_grib: https://ai4edataeuwest.blob.core.windows.net/ecm... local_grib: /home/blaylock/data/aifs/20240611/subset_f812da9... search: :t:850
Copyright statement: Copyright βΒ© 2022 European Centre for Medium-Range Weather Forecasts (ECMWF)β.
Source www.ecmwf.int
Licence Statement: This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/
Disclaimer: ECMWF does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.