ECMWF (IFS and AIFS)#

This tutorial demonstrates how to download the ECMWF Open Data Integrated Forecast System (IFS) and Artificial Intelligence IFS (AIFS).

Recent and archived IFS and AIFS data is freely available from ECMWF in GRIB2 format (πŸ‘€ Read more).

β€œThe data that are becoming available are based on a range of high-resolution forecasts (HRES – 9 km horizontal resolution) and ensemble forecasts (ENS – 18 km horizontal resolution).
29 February 2024: Update from 0.4 degree resolution to 0.25 degree resolution

Data Availability#

  • Jan 18, 2023, IFS 0.4 degree resolution first available

  • Feb 1, 2024 IFS 0.25 degree resolution first available

  • Feb 1, 2024 AIFS 0.25 degree resolution first available

  • May ?, 2024 IFS 0.4 degree discontinued.

Model Types#

ECMWF provides data for two different models

  1. model="ifs" ECMWF Integrated Forecast System

  2. model="aifs" ECMWF Artificial Intelligence Integrated Forecast System

Data Source#

prioriy=

Data source

Archive Duration

"ecmwf"

ECMWF Open Data Server

last 4 days

"azure"

Microsoft Azure

2022-01-21 to present

"aws"

Amazon Web Services

2023-01-18 to present

Products#

Note: the aifs only has the oper product.

product=

Product Description

Available model runs

"oper"

operational high-resolution forecast, atmospheric fields

00z, 12z,

"wave"

wave forecasts

00z, 12z,

"scda"

short cut-off high-resolution forecast, atmospheric fields (also known a high-frequency products)”,

06z, 18z

"scwv"

short cut-off high-resolution forecast, ocean wave fields (also known a high-frequency products)”,

06z, 18z

"enfo"

ensemble forecast, atmospheric fields

00z, 06z, 12z, 18z

"waef"

ensemble forecast, ocean wave fields,

00z, 06z, 12z, 18z

"mmsf"

multi-model seasonal forecasts fields from the ECMWF model only.

?

Model initialized at 00z, 06z, 12z, 18z, but not all products are available every hour.

[1]:
from herbie import Herbie

import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import numpy as np

from herbie import paint
from herbie.toolbox import EasyMap, pc

Integrated Forecast System (IFS)#

IFS data is only available at 0.4 degree prior to February 1, 2024. After that date, the IFS is available at 0.25 degree resolution.

[2]:
H = Herbie("2024-03-1", model="ifs", product="oper", fxx=12)

H.grib, H.idx
βœ… Found β”Š model=ifs β”Š product=oper β”Š 2024-Mar-01 00:00 UTC F12 β”Š GRIB2 @ azure β”Š IDX @ azure
[2]:
('https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2',
 'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.index')
[3]:
# Show the inventory
H.inventory()
[3]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
0 1 0 798588 0-798588 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 tp NaN sfc NaN g 0001 od fc oper :tp:sfc:g:0001:od:fc:oper
1 2 798588 1327999 798588-1327999 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 r 500 pl NaN g 0001 od fc oper :r:500:pl:g:0001:od:fc:oper
2 3 1327999 1817293 1327999-1817293 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 gh 850 pl NaN g 0001 od fc oper :gh:850:pl:g:0001:od:fc:oper
3 4 1817293 2534675 1817293-2534675 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 u 925 pl NaN g 0001 od fc oper :u:925:pl:g:0001:od:fc:oper
4 5 2534675 3268614 2534675-3268614 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 v 925 pl NaN g 0001 od fc oper :v:925:pl:g:0001:od:fc:oper
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
78 79 58285210 59656192 58285210-59656192 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 d 250 pl NaN g 0001 od fc oper :d:250:pl:g:0001:od:fc:oper
79 80 59656192 59759143 59656192-59759143 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 ro NaN sfc NaN g 0001 od fc oper :ro:sfc:g:0001:od:fc:oper
80 81 59759143 61078909 59759143-61078909 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 vo 250 pl NaN g 0001 od fc oper :vo:250:pl:g:0001:od:fc:oper
81 82 61078909 62399470 61078909-62399470 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 d 50 pl NaN g 0001 od fc oper :d:50:pl:g:0001:od:fc:oper
82 83 62399470 63520541 62399470-63520541 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 vo 50 pl NaN g 0001 od fc oper :vo:50:pl:g:0001:od:fc:oper

83 rows Γ— 17 columns

[5]:
# Show just 10-m U and V wind
H.inventory(":10[u|v]:")
[5]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
46 47 28201794 29068335 28201794-29068335 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 10u NaN sfc NaN g 0001 od fc oper :10u:sfc:g:0001:od:fc:oper
47 48 29068335 29928681 29068335-29928681 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 10v NaN sfc NaN g 0001 od fc oper :10v:sfc:g:0001:od:fc:oper
[6]:
# Get 2-m temperature as an xarray Dataset
ds = H.xarray(":2t:", verbose=True)
ds
πŸ‘¨πŸ»β€πŸ­ Created directory: [/home/blaylock/data/ifs/20240301]
πŸ“‡ Download subset: β–Œβ–ŒHerbie IFS model oper product initialized 2024-Mar-01 00:00 UTC F12 β”Š source=azure
 cURL from https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2
Found 1 grib messages.
Download subset group 1
  41  :2t:sfc:g:0001:od:fc:oper
curl -s --range 24427114-25090915 "https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240301/00z/ifs/0p25/oper/20240301000000-12h-oper-fc.grib2" > "/home/blaylock/data/ifs/20240301/subset_e0127a9f__20240301000000-12h-oper-fc.grib2"
πŸ’Ύ Saved the subset to /home/blaylock/data/ifs/20240301/subset_e0127a9f__20240301000000-12h-oper-fc.grib2
[6]:
<xarray.Dataset> Size: 4MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-03-01
    step                 timedelta64[ns] 8B 12:00:00
    heightAboveGround    float64 8B 2.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8
    valid_time           datetime64[ns] 8B 2024-03-01T12:00:00
    gribfile_projection  object 8B None
Data variables:
    t2m                  (latitude, longitude) float32 4MB 244.9 244.9 ... 224.6
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 oper
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20240301/subset_e0127a9f...
    search:                  :2t:
[7]:
ds.t2m.plot()
[7]:
<matplotlib.collections.QuadMesh at 0x7fb7ef104da0>
../../_images/gallery_ecmwf_models_ecmwf_7_1.png

0.4 degree IFS#

Again, the 0.4 degree IFS was available starting January 2023, but will be discontinued in May 2024 in favor of the 0.25 degree data. You can still access these files as long as they exist.

[8]:
# Accessing historical
H = Herbie("2023-07-04", model="ifs", product="oper", fxx=12)
H.grib
βœ… Found β”Š model=ifs β”Š product=oper β”Š 2023-Jul-04 00:00 UTC F12 β”Š GRIB2 @ azure β”Š IDX @ azure
[8]:
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20230704/00z/0p4-beta/oper/20230704000000-12h-oper-fc.grib2'

Eccodes-style Index Files#

The ECMWF index files are different than the wgrib2-style index files, so pay close attention to how you should select the field you want.

[9]:
H = Herbie("2024-03-1", model="ifs", product="oper", fxx=12)

# Show the search_help
print(H.search_help)
βœ… Found β”Š model=ifs β”Š product=oper β”Š 2024-Mar-01 00:00 UTC F12 β”Š GRIB2 @ azure β”Š IDX @ azure

Use regular expression to search for lines in the index file.
Here are some examples you can use for the ecCodes-style `search`

Look at the ECMWF GRIB Parameter Database
https://apps.ecmwf.int/codes/grib/param-db


product=`oper` or `enfo`
======================== ==============================================
search=                  GRIB messages that will be downloaded
======================== ==============================================
":2t:"                   2-m temperature
":2d:"                   2-m dew point temperature
":10u:"                  10-m u wind vector
":10v:"                  10-m v wind vector
":10[uv]:                10-m u and 10-m v wind
":[tuvr]:"               Temp, u/v wind, RH (all levels)
":500:"                  All variables on the 500 hPa level
":gh:500"                Geopotential height only at 500 hPa
":gh:"                   Geopotential height (all pressure levels)
":t:"                    Temperature (all pressure levels)
":q:"                    Specific Humidity (all pressure levels)
":r:"                    Relative humidity (all pressure levels)
":v:"                    v wind vector (all pressure levels)
":u:"                    u wind vector (all pressure levels)
":w:"                    Vertical velocity (Pascals per second)
":lsm:"                  Land-sea mask
":ttr:"                  Top net long-wave (thermal) radiation
":ssrd:"                 Surface short-wave (solar) radiation downwards
":ssr:"                  Surface net short-wave (solar) radiation
":strd:"                 Surface long-wave (thermal) radiation downwards
":str:"                  Surface net long-wave (thermal) radiation
":swvl1:"                Volumetric soil water layer 1 (depth 0 meters)
":swvl2:"                Volumetric soil water layer 2 (depth 7 meters)
":swvl3:"                Volumetric soil water layer 3 (depth 28 meters)
":swvl4:"                Volumetric soil water layer 4 (depth 100 meters)
":skt:"                  Skin (surface) temperature
":d:"                    Divergence (all levels)
":st:"                   Soil temperature
":stl2:"                 Soil temperature level 2 (depth 7 meters)
":tp:"                   Total precipitation
":ro:"                   Run-off
":asn:"                  Snow albedo
":msl:"                  Mean sea level pressure
":sp:"                   Surface pressure
":cape:"                 CAPE
":tcwv:"                 Total column vertically integrated water vapor
":vo:"                   Relative vorticity

product=`wave` or `waef`
======================== ==============================================
search=                  GRIB messages that will be downloaded
======================== ==============================================
":swh:"                  Significant height of wind waves + swell
":mwp:"                  Mean wave period
":mwd:"                  Mean wave direction
":pp1d:"                 Peak wave period
":mp2:"                  Mean zero-crossing wave period

If you need help with regular expression, search the web or look at
this cheatsheet: https://www.petefreitag.com/cheatsheets/regex/.

Here is an example: https://regex101.com/r/niNjwp/1

When considering search queries, pay attention to the β€œsearch_this” column; that columns is used for the regex search.

[10]:
H.inventory()
[10]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
0 1 0 798588 0-798588 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 tp NaN sfc NaN g 0001 od fc oper :tp:sfc:g:0001:od:fc:oper
1 2 798588 1327999 798588-1327999 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 r 500 pl NaN g 0001 od fc oper :r:500:pl:g:0001:od:fc:oper
2 3 1327999 1817293 1327999-1817293 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 gh 850 pl NaN g 0001 od fc oper :gh:850:pl:g:0001:od:fc:oper
3 4 1817293 2534675 1817293-2534675 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 u 925 pl NaN g 0001 od fc oper :u:925:pl:g:0001:od:fc:oper
4 5 2534675 3268614 2534675-3268614 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 v 925 pl NaN g 0001 od fc oper :v:925:pl:g:0001:od:fc:oper
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
78 79 58285210 59656192 58285210-59656192 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 d 250 pl NaN g 0001 od fc oper :d:250:pl:g:0001:od:fc:oper
79 80 59656192 59759143 59656192-59759143 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 ro NaN sfc NaN g 0001 od fc oper :ro:sfc:g:0001:od:fc:oper
80 81 59759143 61078909 59759143-61078909 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 vo 250 pl NaN g 0001 od fc oper :vo:250:pl:g:0001:od:fc:oper
81 82 61078909 62399470 61078909-62399470 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 d 50 pl NaN g 0001 od fc oper :d:50:pl:g:0001:od:fc:oper
82 83 62399470 63520541 62399470-63520541 2024-03-01 2024-03-01 12:00:00 0 days 12:00:00 vo 50 pl NaN g 0001 od fc oper :vo:50:pl:g:0001:od:fc:oper

83 rows Γ— 17 columns

[18]:
H.inventory().param.unique()
[18]:
array(['tp', 'msl', '2t', '10u', 'gh', 't', 'u', 'v', 'st', 'r', 'skt',
       'sp', 'lsm', '10v', 'tcwv', 'q', 'd', 'vo', 'ro'], dtype=object)

Ok, now that we have some understanding of the index file, we can read the 2-m temperature data.

[11]:
ds = H.xarray(":2t:")
ds
[11]:
<xarray.Dataset> Size: 4MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-03-01
    step                 timedelta64[ns] 8B 12:00:00
    heightAboveGround    float64 8B 2.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8
    valid_time           datetime64[ns] 8B 2024-03-01T12:00:00
    gribfile_projection  object 8B None
Data variables:
    t2m                  (latitude, longitude) float32 4MB 244.9 244.9 ... 224.6
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 oper
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20240301/subset_e0127a9f...
    search:                  :2t:
[12]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
    ds.longitude,
    ds.latitude,
    ds.t2m - 273.15,
    transform=pc,
    **paint.NWSTemperature.kwargs2,
)
plt.colorbar(
    p, ax=ax, orientation="horizontal", pad=0.05, **paint.NWSTemperature.cbar_kwargs2
)

ax.set_title(
    f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
    loc="left",
)
ax.set_title(ds.t2m.GRIB_name, loc="right")
[12]:
Text(1.0, 1.0, '2 metre temperature')
../../_images/gallery_ecmwf_models_ecmwf_17_1.png

Now the same, but for wind. Use the Herbie accessor to compute wind speed and direction.

[14]:
H = Herbie("2024-03-01", model="ifs", product="oper")

# Get u and v wind component
ds = H.xarray(":10[u|v]:").herbie.with_wind()
ds
βœ… Found β”Š model=ifs β”Š product=oper β”Š 2024-Mar-01 00:00 UTC F00 β”Š GRIB2 @ azure β”Š IDX @ azure
[14]:
<xarray.Dataset> Size: 17MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-03-01
    step                 timedelta64[ns] 8B 00:00:00
    heightAboveGround    float64 8B 10.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8
    valid_time           datetime64[ns] 8B 2024-03-01
    gribfile_projection  object 8B None
Data variables:
    u10                  (latitude, longitude) float32 4MB 1.654 ... -1.237
    v10                  (latitude, longitude) float32 4MB 0.9546 ... 2.361
    si10                 (latitude, longitude) float32 4MB 1.909 1.909 ... 2.665
    wdir10               (latitude, longitude) float32 4MB 240.0 240.0 ... 152.3
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 oper
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20240301/subset_e0ef1f8f...
    search:                  :10[u|v]:
[16]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
    ds.longitude, ds.latitude, ds.si10, transform=pc, **paint.NWSWindSpeed.kwargs2
)
plt.colorbar(
    p, ax=ax, orientation="horizontal", pad=0.05, **paint.NWSWindSpeed.cbar_kwargs2
)

ax.set_title(
    f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
    loc="left",
)
ax.set_title(ds.si10.standard_name, loc="right")
[16]:
Text(1.0, 1.0, 'wind_speed')
../../_images/gallery_ecmwf_models_ecmwf_20_1.png

Now lets get the humidity and geopotential height at 500 hPa

[17]:
ds = H.xarray(":(?:q|gh):500")
ds
[17]:
<xarray.Dataset> Size: 8MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-03-01
    step                 timedelta64[ns] 8B 00:00:00
    isobaricInhPa        float64 8B 500.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8
    valid_time           datetime64[ns] 8B 2024-03-01
    gribfile_projection  object 8B None
Data variables:
    q                    (latitude, longitude) float32 4MB 0.0001736 ... 0.00...
    gh                   (latitude, longitude) float32 4MB 5.196e+03 ... 4.89...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 oper
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20240301/subset_e0efbf07...
    search:                  :(?:q|gh):500
[19]:
# NOTE: transforming data to a Robinson projection can take some time...
ax = EasyMap("50m", crs=ccrs.Robinson(), figsize=[10, 10]).STATES().BORDERS().ax


# Color shade by specific humidity
p = ax.pcolormesh(ds.longitude, ds.latitude, ds.q, transform=pc, cmap="Greens")

plt.colorbar(
    p,
    ax=ax,
    orientation="horizontal",
    pad=0.05,
    label=f"{ds.q.GRIB_name} ({ds.q.units})",
)

# Contours for geopotential height
ax.contour(
    ds.longitude,
    ds.latitude,
    ds.gh,
    transform=pc,
    colors="k",
    linewidths=0.5,
    levels=range(0, 10_000, 60 * 2),
)


ax.set_title(
    f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
    loc="left",
)
ax.set_title(
    f"{ds.isobaricInhPa.item()} {ds.isobaricInhPa.units}\n{ds.q.GRIB_name}/{ds.gh.GRIB_name}",
    loc="right",
)
[19]:
Text(1.0, 1.0, '500.0 hPa\nSpecific humidity/Geopotential height')
../../_images/gallery_ecmwf_models_ecmwf_23_1.png
The Kernel crashed while executing code in the current cell or a previous cell.

Please review the code in the cell(s) to identify a possible cause of the failure.

Click <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info.

View Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details.

ECMWF IFS Wave Output#

[2]:
H = Herbie("2022-01-26 00:00", model="ifs", product="wave")
βœ… Found β”Š model=ifs β”Š product=wave β”Š 2022-Jan-26 00:00 UTC F00 β”Š GRIB2 @ azure β”Š IDX @ azure
[3]:
ds = H.xarray(None, verbose=True)
ds
/home/blaylock/GITHUB/Herbie/herbie/core.py:1102: UserWarning: Will not remove GRIB file because Herbie will only remove subsetted files (not full files).
  warnings.warn(
βœ… Success! Downloaded IFS from azure               
        src: https://ai4edataeuwest.blob.core.windows.net/ecmwf/20220126/00z/0p4-beta/wave/20220126000000-0h-wave-fc.grib2
        dst: /home/blaylock/data/ifs/20220126/20220126000000-0h-wave-fc.grib2
[3]:
<xarray.Dataset> Size: 8MB
Dimensions:              (latitude: 451, longitude: 900)
Coordinates:
    time                 datetime64[ns] 8B 2022-01-26
    step                 timedelta64[ns] 8B 00:00:00
    meanSea              float64 8B 0.0
  * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
  * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
    valid_time           datetime64[ns] 8B 2022-01-26
    gribfile_projection  object 8B None
Data variables:
    mp2                  (latitude, longitude) float32 2MB ...
    swh                  (latitude, longitude) float32 2MB ...
    mwd                  (latitude, longitude) float32 2MB ...
    pp1d                 (latitude, longitude) float32 2MB ...
    mwp                  (latitude, longitude) float32 2MB ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 wave
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             /home/blaylock/data/ifs/20220126/20220126000000-...
    local_grib:              /home/blaylock/data/ifs/20220126/20220126000000-...
    search:                  None
[4]:
ax = EasyMap("50m", crs=ds.herbie.crs, figsize=[10, 10]).STATES().BORDERS().ax
p = ax.pcolormesh(
    ds.longitude,
    ds.latitude,
    ds.swh,
    transform=pc,
    **paint.NWSWaveHeight.kwargs2,
)
plt.colorbar(
    p,
    ax=ax,
    orientation="horizontal",
    pad=0.05,
    **paint.NWSWaveHeight.cbar_kwargs2,
)

ax.set_title(
    f"{ds.model.upper()}: {H.product_description}\nValid: {ds.valid_time.dt.strftime('%H:%M UTC %d %b %Y').item()}",
    loc="left",
)
ax.set_title(ds.swh.GRIB_name, loc="right")
[4]:
Text(1.0, 1.0, 'Significant height of combined wind waves and swell')
../../_images/gallery_ecmwf_models_ecmwf_27_1.png

Ensemble Forecast Products#

[5]:
H = Herbie("2022-01-26 00:00", model="ifs", product="enfo")
ds = H.xarray(":2t:")
ds
βœ… Found β”Š model=ifs β”Š product=enfo β”Š 2022-Jan-26 00:00 UTC F00 β”Š GRIB2 @ azure β”Š IDX @ azure
Note: Returning a list of [2] xarray.Datasets because cfgrib opened with multiple hypercubes.
[5]:
[<xarray.Dataset> Size: 81MB
 Dimensions:              (number: 50, latitude: 451, longitude: 900)
 Coordinates:
   * number               (number) int64 400B 1 2 3 4 5 6 7 ... 45 46 47 48 49 50
     time                 datetime64[ns] 8B 2022-01-26
     step                 timedelta64[ns] 8B 00:00:00
     heightAboveGround    float64 8B 2.0
   * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
   * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
     valid_time           datetime64[ns] 8B 2022-01-26
     gribfile_projection  object 8B None
 Data variables:
     t2m                  (number, latitude, longitude) float32 81MB 246.5 ......
 Attributes:
     GRIB_edition:            2
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts
     model:                   ifs
     product:                 enfo
     description:             ECMWF Open Data - Integrated Forecast System
     remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
     local_grib:              /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
     search:                  :2t:,
 <xarray.Dataset> Size: 2MB
 Dimensions:              (latitude: 451, longitude: 900)
 Coordinates:
     number               int64 8B 0
     time                 datetime64[ns] 8B 2022-01-26
     step                 timedelta64[ns] 8B 00:00:00
     heightAboveGround    float64 8B 2.0
   * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
   * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
     valid_time           datetime64[ns] 8B 2022-01-26
     gribfile_projection  object 8B None
 Data variables:
     t2m                  (latitude, longitude) float32 2MB 246.2 246.2 ... 246.3
 Attributes:
     GRIB_edition:            2
     GRIB_centre:             ecmf
     GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
     GRIB_subCentre:          0
     Conventions:             CF-1.7
     institution:             European Centre for Medium-Range Weather Forecasts
     model:                   ifs
     product:                 enfo
     description:             ECMWF Open Data - Integrated Forecast System
     remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
     local_grib:              /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
     search:                  :2t:]
[6]:
# Dataset with all 50 members
ds[0]
[6]:
<xarray.Dataset> Size: 81MB
Dimensions:              (number: 50, latitude: 451, longitude: 900)
Coordinates:
  * number               (number) int64 400B 1 2 3 4 5 6 7 ... 45 46 47 48 49 50
    time                 datetime64[ns] 8B 2022-01-26
    step                 timedelta64[ns] 8B 00:00:00
    heightAboveGround    float64 8B 2.0
  * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
  * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
    valid_time           datetime64[ns] 8B 2022-01-26
    gribfile_projection  object 8B None
Data variables:
    t2m                  (number, latitude, longitude) float32 81MB 246.5 ......
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 enfo
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
    search:                  :2t:
[7]:
# This Dataset is of the mean of all the members, right?
ds[1]
[7]:
<xarray.Dataset> Size: 2MB
Dimensions:              (latitude: 451, longitude: 900)
Coordinates:
    number               int64 8B 0
    time                 datetime64[ns] 8B 2022-01-26
    step                 timedelta64[ns] 8B 00:00:00
    heightAboveGround    float64 8B 2.0
  * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
  * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
    valid_time           datetime64[ns] 8B 2022-01-26
    gribfile_projection  object 8B None
Data variables:
    t2m                  (latitude, longitude) float32 2MB 246.2 246.2 ... 246.3
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 enfo
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20220126/subset_bfef7f9f...
    search:                  :2t:
[8]:
H.idx
[8]:
'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20220126/00z/0p4-beta/enfo/20220126000000-0h-enfo-ef.index'

Ensemble Wave Products#

[16]:
H = Herbie("2022-01-26 00:00", model="ifs", product="waef")
H.inventory()
βœ… Found β”Š model=ifs β”Š product=waef β”Š 2022-Jan-26 00:00 UTC F00 β”Š GRIB2 @ azure β”Š IDX @ azure
[16]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
0 1 0 501693 0-501693 2022-01-26 2022-01-26 0 days swh NaN sfc 38 g 0001 od pf waef :swh:sfc:38:g:0001:od:pf:waef
1 2 501693 1003064 501693-1003064 2022-01-26 2022-01-26 0 days swh NaN sfc 46 g 0001 od pf waef :swh:sfc:46:g:0001:od:pf:waef
2 3 1003064 1504757 1003064-1504757 2022-01-26 2022-01-26 0 days swh NaN sfc 43 g 0001 od pf waef :swh:sfc:43:g:0001:od:pf:waef
3 4 1504757 2006096 1504757-2006096 2022-01-26 2022-01-26 0 days swh NaN sfc 40 g 0001 od pf waef :swh:sfc:40:g:0001:od:pf:waef
4 5 2006096 2507435 2006096-2507435 2022-01-26 2022-01-26 0 days swh NaN sfc NaN g 0001 od cf waef :swh:sfc:g:0001:od:cf:waef
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
250 251 125340988 125842349 125340988-125842349 2022-01-26 2022-01-26 0 days pp1d NaN sfc 7 g 0001 od pf waef :pp1d:sfc:7:g:0001:od:pf:waef
251 252 125842349 126343438 125842349-126343438 2022-01-26 2022-01-26 0 days pp1d NaN sfc 29 g 0001 od pf waef :pp1d:sfc:29:g:0001:od:pf:waef
252 253 126343438 126844527 126343438-126844527 2022-01-26 2022-01-26 0 days pp1d NaN sfc 34 g 0001 od pf waef :pp1d:sfc:34:g:0001:od:pf:waef
253 254 126844527 127345908 126844527-127345908 2022-01-26 2022-01-26 0 days pp1d NaN sfc NaN g 0001 od cf waef :pp1d:sfc:g:0001:od:cf:waef
254 255 127345908 127847247 127345908-127847247 2022-01-26 2022-01-26 0 days mp2 NaN sfc NaN g 0001 od cf waef :mp2:sfc:g:0001:od:cf:waef

255 rows Γ— 17 columns

[18]:
H = Herbie("2022-01-26", model="ifs", product="enfo")
H.inventory()
βœ… Found β”Š model=ifs β”Š product=enfo β”Š 2022-Jan-26 00:00 UTC F00 β”Š GRIB2 @ azure β”Š IDX @ azure
[18]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
0 1 0 609069 0-609069 2022-01-26 2022-01-26 0 days 2t NaN sfc 29 g 0001 od pf enfo :2t:sfc:29:g:0001:od:pf:enfo
1 2 609069 1218138 609069-1218138 2022-01-26 2022-01-26 0 days 10u NaN sfc 27 g 0001 od pf enfo :10u:sfc:27:g:0001:od:pf:enfo
2 3 1218138 1827207 1218138-1827207 2022-01-26 2022-01-26 0 days 10v NaN sfc 14 g 0001 od pf enfo :10v:sfc:14:g:0001:od:pf:enfo
3 4 1827207 2436276 1827207-2436276 2022-01-26 2022-01-26 0 days 2t NaN sfc 2 g 0001 od pf enfo :2t:sfc:2:g:0001:od:pf:enfo
4 5 2436276 3045345 2436276-3045345 2022-01-26 2022-01-26 0 days 10u NaN sfc 43 g 0001 od pf enfo :10u:sfc:43:g:0001:od:pf:enfo
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
4177 4178 2574120261 2574932280 2574120261-2574932280 2022-01-26 2022-01-26 0 days d 200 pl 20 g 0001 od pf enfo :d:200:pl:20:g:0001:od:pf:enfo
4178 4179 2574932280 2575744299 2574932280-2575744299 2022-01-26 2022-01-26 0 days d 200 pl 41 g 0001 od pf enfo :d:200:pl:41:g:0001:od:pf:enfo
4179 4180 2575744299 2576556318 2575744299-2576556318 2022-01-26 2022-01-26 0 days d 200 pl 4 g 0001 od pf enfo :d:200:pl:4:g:0001:od:pf:enfo
4180 4181 2576556318 2577368337 2576556318-2577368337 2022-01-26 2022-01-26 0 days d 200 pl 50 g 0001 od pf enfo :d:200:pl:50:g:0001:od:pf:enfo
4181 4182 2577368337 2578180356 2577368337-2578180356 2022-01-26 2022-01-26 0 days d 200 pl 23 g 0001 od pf enfo :d:200:pl:23:g:0001:od:pf:enfo

4182 rows Γ— 17 columns

Here is another examle, just for fun

[21]:
H = Herbie("2022-01-26", model="ifs", product="oper", fxx=12)
H
βœ… Found β”Š model=ifs β”Š product=oper β”Š 2022-Jan-26 00:00 UTC F12 β”Š GRIB2 @ azure β”Š IDX @ azure
[21]:
β–Œβ–ŒHerbie IFS model oper product initialized 2022-Jan-26 00:00 UTC F12 β”Š source=azure
[22]:
# Download just the 10-m u and v winds
ds = H.download(search=":10(u|v):")
ds
/home/blaylock/GITHUB/Herbie/herbie/core.py:808: UserWarning: This pattern is interpreted as a regular expression, and has match groups. To actually get the groups, use str.extract.
  logic = df.search_this.str.contains(search)
[22]:
PosixPath('/home/blaylock/data/ifs/20220126/subset_bf121a77__20220126000000-12h-oper-fc.grib2')
[23]:
# Retrieve the 500 hPa temperature as an xarray.Dataset
ds = H.xarray(search=":t:500:")
ds
[23]:
<xarray.Dataset> Size: 2MB
Dimensions:              (latitude: 451, longitude: 900)
Coordinates:
    time                 datetime64[ns] 8B 2022-01-26
    step                 timedelta64[ns] 8B 12:00:00
    isobaricInhPa        float64 8B 500.0
  * latitude             (latitude) float64 4kB 90.0 89.6 89.2 ... -89.6 -90.0
  * longitude            (longitude) float64 7kB -180.0 -179.6 ... 179.2 179.6
    valid_time           datetime64[ns] 8B 2022-01-26T12:00:00
    gribfile_projection  object 8B None
Data variables:
    t                    (latitude, longitude) float32 2MB 234.9 234.9 ... 242.5
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   ifs
    product:                 oper
    description:             ECMWF Open Data - Integrated Forecast System
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/ifs/20220126/subset_bf124169...
    search:                  :t:500:
[24]:
ds.t.plot()
[24]:
<matplotlib.collections.QuadMesh at 0x7f756deb8500>
../../_images/gallery_ecmwf_models_ecmwf_40_1.png

Artificial Intelligence IFS#

Download and plot ECMWF AIFS model data.

⚠ AIFS GRIB files before March 13, 2024, seemed to be encoded incorrectly as they are read as a vector instead of a grid. I’m not sure how this data can be recovered or reshaped.

[29]:
H = Herbie("2024-06-11", model="aifs", product="oper", fxx=12)

H.grib, H.idx
βœ… Found β”Š model=aifs β”Š product=oper β”Š 2024-Jun-11 00:00 UTC F12 β”Š GRIB2 @ azure β”Š IDX @ azure
[29]:
('https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240611/00z/aifs/0p25/oper/20240611000000-12h-oper-fc.grib2',
 'https://ai4edataeuwest.blob.core.windows.net/ecmwf/20240611/00z/aifs/0p25/oper/20240611000000-12h-oper-fc.index')
[30]:
H.inventory(":t:850")
[30]:
grib_message start_byte end_byte range reference_time valid_time step param levelist levtype number domain expver class type stream search_this
32 33 25515811 26027206 25515811-26027206 2024-06-11 2024-06-11 12:00:00 0 days 12:00:00 t 850 pl NaN g 0001 ai fc oper :t:850:pl:g:0001:ai:fc:oper
[42]:
ds = H.xarray(":t:850")
(ds.t - 273.15).plot(
    x="longitude",
    y="latitude",
    ax=EasyMap().ax,
    transform=pc,
    **paint.NWSTemperature.kwargs2,
    cbar_kwargs=paint.NWSTemperature.cbar_kwargs2
    | dict(orientation="horizontal", pad=0.01),
)
ds
[42]:
<xarray.Dataset> Size: 4MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-06-11
    step                 timedelta64[ns] 8B 12:00:00
    isobaricInhPa        float64 8B 850.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB -180.0 -179.8 ... 179.5 179.8
    valid_time           datetime64[ns] 8B 2024-06-11T12:00:00
    gribfile_projection  object 8B None
Data variables:
    t                    (latitude, longitude) float32 4MB 271.1 271.1 ... 235.1
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    model:                   aifs
    product:                 oper
    description:             ECMWF Open Data - Artificial Inteligence Integra...
    remote_grib:             https://ai4edataeuwest.blob.core.windows.net/ecm...
    local_grib:              /home/blaylock/data/aifs/20240611/subset_f812da9...
    search:                  :t:850
../../_images/gallery_ecmwf_models_ecmwf_44_1.png

Attribution

  • Copyright statement: Copyright β€œΒ© 2022 European Centre for Medium-Range Weather Forecasts (ECMWF)”.

  • Source www.ecmwf.int

  • Licence Statement: This data is published under a Creative Commons Attribution 4.0 International (CC BY 4.0). https://creativecommons.org/licenses/by/4.0/

  • Disclaimer: ECMWF does not accept any liability whatsoever for any error or omission in the data, their availability, or for any loss or damage arising from their use.