🚦 “Start your engines!”

With Herbie installed, you are about to begin the race. The following are some key concepts:

1. Numerical weather prediction (NWP) data is diseminated in GRIB2 format.

2. Herbie helps you find, download, and load NWP data from different archive sources.

3. The Herbie class is the most fundamental component of Herbie. Herbie races to look for the NWP model data you ask for.

Let’s see how to use Herbe. First, import Herbie.

from herbie import Herbie

ERROR 1: PROJ: proj_create_from_database: Open of /home/blaylock/miniconda3/envs/herbie-dev/share/proj failed

Tell Herbie the model run date you are interested in. The input date can be a Pandas parseable Date-Time string, or a datetime object.

By default it looks for the HRRR model (unless you modified the Herbie configuration file).

When I create Herbie objects, I like to save them to the variable H:

H = Herbie("2024-04-01 12:00")

✅ Found ┊ model=hrrr ┊ product=sfc ┊ 2024-Apr-01 12:00 UTC F00 ┊ GRIB2 @ aws ┊ IDX @ aws

You can see that Herbie says it found the HRRR model for the date we asked for, at Amazon Web Services (AWS).

Show where the file is located:

H.grib

'https://noaa-hrrr-bdp-pds.s3.amazonaws.com/hrrr.20240401/conus/hrrr.t12z.wrfsfcf00.grib2'

If no file was found, Herbie will tell you.

Herbie("2000-01-01")

💔 Did not find ┊ model=hrrr ┊ product=sfc ┊ 2000-Jan-01 00:00 UTC F00

▌▌Herbie HRRR model sfc product initialized 2000-Jan-01 00:00 UTC F00 ┊ source=None

Ok, back to our example. You can specify other arguments in the Herbie class.

H = Herbie(
    "2024-04-01 12:00",  # Model inititializtion time
    model="hrrr",  # Model name
    fxx=6,  # Forecast step, in hours
    product="sfc",  # Model product
)
H

✅ Found ┊ model=hrrr ┊ product=sfc ┊ 2024-Apr-01 12:00 UTC F06 ┊ GRIB2 @ aws ┊ IDX @ aws

▌▌Herbie HRRR model sfc product initialized 2024-Apr-01 12:00 UTC F06 ┊ source=aws

There are three fundamental things you can do with a Herbie object:

1. Show an inventory of the file contents;

2. Download the GRIB2 file to your local machine;

3. Open specific GRIB messages (variables) into an xarray Dataset.

GRIB2 file inventory

File invenotries are retuned as a Pandas DataFrame.

Let’s first show how to show a file inventory for a GFS 12-hr forecast file.

H = Herbie("2024-02-05", model="gfs", fxx=12)
H.inventory()

✅ Found ┊ model=gfs ┊ product=pgrb2.0p25 ┊ 2024-Feb-05 00:00 UTC F12 ┊ GRIB2 @ aws ┊ IDX @ aws

	grib_message	start_byte	end_byte	range	reference_time	valid_time	variable	level	forecast_time	search_this
0	1	0	999868.0	0-999868	2024-02-05	2024-02-05 12:00:00	PRMSL	mean sea level	12 hour fcst	:PRMSL:mean sea level:12 hour fcst
1	2	999869	1091477.0	999869-1091477	2024-02-05	2024-02-05 12:00:00	CLMR	1 hybrid level	12 hour fcst	:CLMR:1 hybrid level:12 hour fcst
2	3	1091478	1376295.0	1091478-1376295	2024-02-05	2024-02-05 12:00:00	ICMR	1 hybrid level	12 hour fcst	:ICMR:1 hybrid level:12 hour fcst
3	4	1376296	1621422.0	1376296-1621422	2024-02-05	2024-02-05 12:00:00	RWMR	1 hybrid level	12 hour fcst	:RWMR:1 hybrid level:12 hour fcst
4	5	1621423	1717851.0	1621423-1717851	2024-02-05	2024-02-05 12:00:00	SNMR	1 hybrid level	12 hour fcst	:SNMR:1 hybrid level:12 hour fcst
...	...	...	...	...	...	...	...	...	...	...
738	739	534226517	534874064.0	534226517-534874064	2024-02-05	2024-02-05 12:00:00	VGRD	PV=-2e-06 (Km^2/kg/s) surface	12 hour fcst	:VGRD:PV=-2e-06 (Km^2/kg/s) surface:12 hour fcst
739	740	534874065	535523722.0	534874065-535523722	2024-02-05	2024-02-05 12:00:00	TMP	PV=-2e-06 (Km^2/kg/s) surface	12 hour fcst	:TMP:PV=-2e-06 (Km^2/kg/s) surface:12 hour fcst
740	741	535523723	536688853.0	535523723-536688853	2024-02-05	2024-02-05 12:00:00	HGT	PV=-2e-06 (Km^2/kg/s) surface	12 hour fcst	:HGT:PV=-2e-06 (Km^2/kg/s) surface:12 hour fcst
741	742	536688854	537810720.0	536688854-537810720	2024-02-05	2024-02-05 12:00:00	PRES	PV=-2e-06 (Km^2/kg/s) surface	12 hour fcst	:PRES:PV=-2e-06 (Km^2/kg/s) surface:12 hour fcst
742	743	537810721	NaN	537810721-	2024-02-05	2024-02-05 12:00:00	VWSH	PV=-2e-06 (Km^2/kg/s) surface	12 hour fcst	:VWSH:PV=-2e-06 (Km^2/kg/s) surface:12 hour fcst

743 rows × 10 columns

GRIB files have a lot of data in them. Each GRIB message represents a different variable at a specific level or layer. You can filter the GRIB messages for a specific variable. Finding exactly what you want takes some getting used to, but if you play with the inventory print out, you’ll get the hange of it.

Pay special attention to the search_this column; you can use regular expression to filter the rows based on that column.

Don’t know regex yet? It is worth learning, I highly recomend it 😄 You can use sites like https://regex101.com/ or ChatGPT to explain how regex works.

For example, lets get all temperature fields on pressure levels: (Explain)

Note: the r preceeding r"string" means this is a “raw” string, which is needed for some regex expressions so backslashes are not escaped. When using regex as a string, it’s best practice to get in the habit of using r"string".

H.inventory(r":TMP:\d+ mb:")

	grib_message	start_byte	end_byte	range	reference_time	valid_time	variable	level	forecast_time	search_this
95	96	74664821	75478525.0	74664821-75478525	2024-02-05	2024-02-05 12:00:00	TMP	1 mb	12 hour fcst	:TMP:1 mb:12 hour fcst
105	106	82369600	83190647.0	82369600-83190647	2024-02-05	2024-02-05 12:00:00	TMP	2 mb	12 hour fcst	:TMP:2 mb:12 hour fcst
115	116	90185842	90990476.0	90185842-90990476	2024-02-05	2024-02-05 12:00:00	TMP	3 mb	12 hour fcst	:TMP:3 mb:12 hour fcst
125	126	97861692	98668588.0	97861692-98668588	2024-02-05	2024-02-05 12:00:00	TMP	5 mb	12 hour fcst	:TMP:5 mb:12 hour fcst
135	136	105711779	106489418.0	105711779-106489418	2024-02-05	2024-02-05 12:00:00	TMP	7 mb	12 hour fcst	:TMP:7 mb:12 hour fcst
145	146	113582833	114350030.0	113582833-114350030	2024-02-05	2024-02-05 12:00:00	TMP	10 mb	12 hour fcst	:TMP:10 mb:12 hour fcst
155	156	121299505	122061980.0	121299505-122061980	2024-02-05	2024-02-05 12:00:00	TMP	15 mb	12 hour fcst	:TMP:15 mb:12 hour fcst
165	166	129508823	130263235.0	129508823-130263235	2024-02-05	2024-02-05 12:00:00	TMP	20 mb	12 hour fcst	:TMP:20 mb:12 hour fcst
175	176	137665804	138402308.0	137665804-138402308	2024-02-05	2024-02-05 12:00:00	TMP	30 mb	12 hour fcst	:TMP:30 mb:12 hour fcst
185	186	146139516	146897181.0	146139516-146897181	2024-02-05	2024-02-05 12:00:00	TMP	40 mb	12 hour fcst	:TMP:40 mb:12 hour fcst
195	196	154782262	155525408.0	154782262-155525408	2024-02-05	2024-02-05 12:00:00	TMP	50 mb	12 hour fcst	:TMP:50 mb:12 hour fcst
211	212	166862208	167619138.0	166862208-167619138	2024-02-05	2024-02-05 12:00:00	TMP	70 mb	12 hour fcst	:TMP:70 mb:12 hour fcst
221	222	175629371	176389735.0	175629371-176389735	2024-02-05	2024-02-05 12:00:00	TMP	100 mb	12 hour fcst	:TMP:100 mb:12 hour fcst
237	238	186173272	186923340.0	186173272-186923340	2024-02-05	2024-02-05 12:00:00	TMP	150 mb	12 hour fcst	:TMP:150 mb:12 hour fcst
253	254	195633620	196392162.0	195633620-196392162	2024-02-05	2024-02-05 12:00:00	TMP	200 mb	12 hour fcst	:TMP:200 mb:12 hour fcst
269	270	205549986	206297215.0	205549986-206297215	2024-02-05	2024-02-05 12:00:00	TMP	250 mb	12 hour fcst	:TMP:250 mb:12 hour fcst
285	286	215971488	216733986.0	215971488-216733986	2024-02-05	2024-02-05 12:00:00	TMP	300 mb	12 hour fcst	:TMP:300 mb:12 hour fcst
301	302	226589468	227325614.0	226589468-227325614	2024-02-05	2024-02-05 12:00:00	TMP	350 mb	12 hour fcst	:TMP:350 mb:12 hour fcst
317	318	236966573	237691529.0	236966573-237691529	2024-02-05	2024-02-05 12:00:00	TMP	400 mb	12 hour fcst	:TMP:400 mb:12 hour fcst
333	334	247327024	248051266.0	247327024-248051266	2024-02-05	2024-02-05 12:00:00	TMP	450 mb	12 hour fcst	:TMP:450 mb:12 hour fcst
349	350	257636441	258358197.0	257636441-258358197	2024-02-05	2024-02-05 12:00:00	TMP	500 mb	12 hour fcst	:TMP:500 mb:12 hour fcst
365	366	267803279	268530231.0	267803279-268530231	2024-02-05	2024-02-05 12:00:00	TMP	550 mb	12 hour fcst	:TMP:550 mb:12 hour fcst
381	382	278606517	279337326.0	278606517-279337326	2024-02-05	2024-02-05 12:00:00	TMP	600 mb	12 hour fcst	:TMP:600 mb:12 hour fcst
397	398	289355302	290094486.0	289355302-290094486	2024-02-05	2024-02-05 12:00:00	TMP	650 mb	12 hour fcst	:TMP:650 mb:12 hour fcst
413	414	300335831	301091467.0	300335831-301091467	2024-02-05	2024-02-05 12:00:00	TMP	700 mb	12 hour fcst	:TMP:700 mb:12 hour fcst
429	430	311417304	312194671.0	311417304-312194671	2024-02-05	2024-02-05 12:00:00	TMP	750 mb	12 hour fcst	:TMP:750 mb:12 hour fcst
445	446	322661406	323469528.0	322661406-323469528	2024-02-05	2024-02-05 12:00:00	TMP	800 mb	12 hour fcst	:TMP:800 mb:12 hour fcst
461	462	334281650	335121092.0	334281650-335121092	2024-02-05	2024-02-05 12:00:00	TMP	850 mb	12 hour fcst	:TMP:850 mb:12 hour fcst
477	478	346331108	347187535.0	346331108-347187535	2024-02-05	2024-02-05 12:00:00	TMP	900 mb	12 hour fcst	:TMP:900 mb:12 hour fcst
493	494	358388768	359246780.0	358388768-359246780	2024-02-05	2024-02-05 12:00:00	TMP	925 mb	12 hour fcst	:TMP:925 mb:12 hour fcst
509	510	370297558	371154825.0	370297558-371154825	2024-02-05	2024-02-05 12:00:00	TMP	950 mb	12 hour fcst	:TMP:950 mb:12 hour fcst
526	527	382154490	383009772.0	382154490-383009772	2024-02-05	2024-02-05 12:00:00	TMP	975 mb	12 hour fcst	:TMP:975 mb:12 hour fcst
541	542	392327983	393187769.0	392327983-393187769	2024-02-05	2024-02-05 12:00:00	TMP	1000 mb	12 hour fcst	:TMP:1000 mb:12 hour fcst

Lets do another.

Get U and V wind at 10 m above the surface: (Explain)

H.inventory(r":[U|V]GRD:10 m above")

	grib_message	start_byte	end_byte	range	reference_time	valid_time	variable	level	forecast_time	search_this
587	588	419775774	420738935.0	419775774-420738935	2024-02-05	2024-02-05 12:00:00	UGRD	10 m above ground	12 hour fcst	:UGRD:10 m above ground:12 hour fcst
588	589	420738936	421677436.0	420738936-421677436	2024-02-05	2024-02-05 12:00:00	VGRD	10 m above ground	12 hour fcst	:VGRD:10 m above ground:12 hour fcst

Download GRIB files

Herbie can download full files or select messages from a GRIB file.

Let’s download a full HRRR file, and for the sake of demonstration, lets turn on verbose information:

H = Herbie("2024-01-01 06:00", model="hrrr", fxx=6)
myFile = H.download(verbose=True)
myFile

✅ Found ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-01 06:00 UTC F06 ┊ GRIB2 @ aws ┊ IDX @ aws
👨🏻‍🏭 Created directory: [/home/blaylock/data/hrrr/20240101]
✅ Success! Downloaded HRRR from aws                 
        src: https://noaa-hrrr-bdp-pds.s3.amazonaws.com/hrrr.20240101/conus/hrrr.t06z.wrfsfcf06.grib2
        dst: /home/blaylock/data/hrrr/20240101/hrrr.t06z.wrfsfcf06.grib2

PosixPath('/home/blaylock/data/hrrr/20240101/hrrr.t06z.wrfsfcf06.grib2')

HRRR files are large–over 100 mb per file–which takes time and space (GFS files are even larger; over 500 MB for a single file!). You very likely don’t need everything that is provided in a GRIB file. This is why knowing how to filter the inventory, as shown earlier, is very important; you can specify which GRIB messages you actually want to download.

Now lets just download the 10-m wind fields from the HRRR file:

H = Herbie("2024-01-01 06:00", model="gfs", fxx=6)
mySubset = H.download(r":[U|V]GRD:10 m above", verbose=True)
mySubset

✅ Found ┊ model=gfs ┊ product=pgrb2.0p25 ┊ 2024-Jan-01 06:00 UTC F06 ┊ GRIB2 @ aws ┊ IDX @ aws
👨🏻‍🏭 Created directory: [/home/blaylock/data/gfs/20240101]
📇 Download subset: ▌▌Herbie GFS model pgrb2.0p25 product initialized 2024-Jan-01 06:00 UTC F06 ┊ source=aws
 cURL from https://noaa-gfs-bdp-pds.s3.amazonaws.com/gfs.20240101/06/atmos/gfs.t06z.pgrb2.0p25.f006
Found 2 grib messages.
Download subset group 1
  588 :UGRD:10 m above ground:6 hour fcst
  589 :VGRD:10 m above ground:6 hour fcst
curl -s --range 417094105-418994003 "https://noaa-gfs-bdp-pds.s3.amazonaws.com/gfs.20240101/06/atmos/gfs.t06z.pgrb2.0p25.f006" > "/home/blaylock/data/gfs/20240101/subset_33b288d0__gfs.t06z.pgrb2.0p25.f006"
💾 Saved the subset to /home/blaylock/data/gfs/20240101/subset_33b288d0__gfs.t06z.pgrb2.0p25.f006

PosixPath('/home/blaylock/data/gfs/20240101/subset_33b288d0__gfs.t06z.pgrb2.0p25.f006')

Downloading the subset was much faster download, and it only takes ~2 MB of space.

Read GRIB into xarray

Herbie uses the cfgrib library to read GRIB data into an xarray Dataset. It is important that you know how to subset the files for variables you want so you only read the data you need.

When loading the data into xarray, Herbie downloads the file temporarily by default, and removes the file when it is loaded into memory. This is the default because I don’t want to hog so much diskspace on a users system, and I assume most users will use the xarray capability for quickly looking at data rather than keeping an archive.

Let’s get the 2-m temperature and dew point from the GFS model.

H = Herbie("2024-01-01", model="gfs")
ds = H.xarray(r":(?:TMP|DPT):2 m above")
ds

✅ Found ┊ model=gfs ┊ product=pgrb2.0p25 ┊ 2024-Jan-01 00:00 UTC F00 ┊ GRIB2 @ aws ┊ IDX @ aws

<xarray.Dataset> Size: 8MB
Dimensions:              (latitude: 721, longitude: 1440)
Coordinates:
    time                 datetime64[ns] 8B 2024-01-01
    step                 timedelta64[ns] 8B 00:00:00
    heightAboveGround    float64 8B 2.0
  * latitude             (latitude) float64 6kB 90.0 89.75 89.5 ... -89.75 -90.0
  * longitude            (longitude) float64 12kB 0.0 0.25 0.5 ... 359.5 359.8
    valid_time           datetime64[ns] 8B 2024-01-01
    gribfile_projection  object 8B None
Data variables:
    t2m                  (latitude, longitude) float32 4MB 244.8 244.8 ... 256.5
    d2m                  (latitude, longitude) float32 4MB 241.8 241.8 ... 254.0
Attributes:
    GRIB_edition:            2
    GRIB_centre:             kwbc
    GRIB_centreDescription:  US National Weather Service - NCEP
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             US National Weather Service - NCEP
    model:                   gfs
    product:                 pgrb2.0p25
    description:             Global Forecast System
    remote_grib:             https://noaa-gfs-bdp-pds.s3.amazonaws.com/gfs.20...
    local_grib:              /home/blaylock/data/gfs/20240101/subset_6befcb50...
    searchString:            :(?:TMP|DPT):2 m above

xarray.Dataset

• Dimensions:
  • latitude: 721
  • longitude: 1440
• Coordinates: (7)
  • time
    ()
    datetime64[ns]
    2024-01-01

    long_name :

    initial time of forecast

    standard_name :

    forecast_reference_time

    array('2024-01-01T00:00:00.000000000', dtype='datetime64[ns]')

  • step
    ()
    timedelta64[ns]
    00:00:00

    long_name :

    time since forecast_reference_time

    standard_name :

    forecast_period

    array(0, dtype='timedelta64[ns]')

  • heightAboveGround
    ()
    float64
    2.0

    long_name :

    height above the surface

    units :

    m

    positive :

    up

    standard_name :

    height

    array(2.)

  • latitude
    (latitude)
    float64
    90.0 89.75 89.5 ... -89.75 -90.0

    units :

    degrees_north

    standard_name :

    latitude

    long_name :

    latitude

    stored_direction :

    decreasing

    array([ 90.  ,  89.75,  89.5 , ..., -89.5 , -89.75, -90.  ])

  • longitude
    (longitude)
    float64
    0.0 0.25 0.5 ... 359.2 359.5 359.8

    units :

    degrees_east

    standard_name :

    longitude

    long_name :

    longitude

    array([0.0000e+00, 2.5000e-01, 5.0000e-01, ..., 3.5925e+02, 3.5950e+02,
           3.5975e+02])

  • valid_time
    ()
    datetime64[ns]
    2024-01-01

    standard_name :

    time

    long_name :

    time

    array('2024-01-01T00:00:00.000000000', dtype='datetime64[ns]')

  • gribfile_projection
    ()
    object
    None

    crs_wkt :

    GEOGCRS["unknown",DATUM["unknown",ELLIPSOID["unknown",6371229,0,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]],CS[ellipsoidal,2],AXIS["longitude",east,ORDER[1],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]],AXIS["latitude",north,ORDER[2],ANGLEUNIT["degree",0.0174532925199433,ID["EPSG",9122]]]]

    semi_major_axis :

    6371229.0

    semi_minor_axis :

    6371229.0

    inverse_flattening :

    0.0

    reference_ellipsoid_name :

    unknown

    longitude_of_prime_meridian :

    0.0

    prime_meridian_name :

    Greenwich

    geographic_crs_name :

    unknown

    horizontal_datum_name :

    unknown

    grid_mapping_name :

    latitude_longitude

    long_name :

    GFS model grid projection

    array(None, dtype=object)

• Data variables: (2)
  • t2m
    (latitude, longitude)
    float32
    244.8 244.8 244.8 ... 256.5 256.5

    GRIB_paramId :

    167

    GRIB_dataType :

    fc

    GRIB_numberOfPoints :

    1038240

    GRIB_typeOfLevel :

    heightAboveGround

    GRIB_stepUnits :

    1

    GRIB_stepType :

    instant

    GRIB_gridType :

    regular_ll

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    air_temperature

    GRIB_cfVarName :

    t2m

    GRIB_gridDefinitionDescription :

    Latitude/longitude. Also called equidistant cylindrical, or Plate Carree

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    0.0

    GRIB_longitudeOfLastGridPointInDegrees :

    359.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_parameterName :

    Temperature

    GRIB_parameterUnits :

    K

    GRIB_shortName :

    2t

    GRIB_stepRange :

    0

    GRIB_units :

    K

    long_name :

    2 metre temperature

    units :

    K

    standard_name :

    air_temperature

    grid_mapping :

    gribfile_projection

    array([[244.79486, 244.79486, 244.79486, ..., 244.79486, 244.79486,
            244.79486],
           [244.88486, 244.88486, 244.87486, ..., 244.88486, 244.88486,
            244.88486],
           [246.35486, 246.34486, 246.34486, ..., 246.35486, 246.35486,
            246.35486],
           ...,
           [254.59486, 254.57486, 254.56487, ..., 254.63486, 254.62486,
            254.60486],
           [255.79486, 255.79486, 255.79486, ..., 255.80487, 255.80487,
            255.79486],
           [256.54486, 256.54486, 256.54486, ..., 256.54486, 256.54486,
            256.54486]], dtype=float32)

  • d2m
    (latitude, longitude)
    float32
    241.8 241.8 241.8 ... 254.0 254.0

    GRIB_paramId :

    168

    GRIB_dataType :

    fc

    GRIB_numberOfPoints :

    1038240

    GRIB_typeOfLevel :

    heightAboveGround

    GRIB_stepUnits :

    1

    GRIB_stepType :

    instant

    GRIB_gridType :

    regular_ll

    GRIB_NV :

    0

    GRIB_Nx :

    1440

    GRIB_Ny :

    721

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    d2m

    GRIB_gridDefinitionDescription :

    Latitude/longitude. Also called equidistant cylindrical, or Plate Carree

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    90.0

    GRIB_latitudeOfLastGridPointInDegrees :

    -90.0

    GRIB_longitudeOfFirstGridPointInDegrees :

    0.0

    GRIB_longitudeOfLastGridPointInDegrees :

    359.75

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre dewpoint temperature

    GRIB_parameterName :

    Dew point temperature

    GRIB_parameterUnits :

    K

    GRIB_shortName :

    2d

    GRIB_stepRange :

    0

    GRIB_units :

    K

    long_name :

    2 metre dewpoint temperature

    units :

    K

    standard_name :

    unknown

    grid_mapping :

    gribfile_projection

    array([[241.82924, 241.82924, 241.82924, ..., 241.82924, 241.82924,
            241.82924],
           [241.91924, 241.91924, 241.91924, ..., 241.91924, 241.91924,
            241.91924],
           [243.51924, 243.51924, 243.50923, ..., 243.52924, 243.52924,
            243.52924],
           ...,
           [251.84924, 251.82924, 251.81924, ..., 251.88924, 251.86923,
            251.85924],
           [253.15924, 253.14923, 253.14923, ..., 253.16924, 253.15924,
            253.15924],
           [253.98924, 253.98924, 253.98924, ..., 253.98924, 253.98924,
            253.98924]], dtype=float32)

• Indexes: (2)
  • latitude
    PandasIndex

    PandasIndex(Index([  90.0,  89.75,   89.5,  89.25,   89.0,  88.75,   88.5,  88.25,   88.0,
            87.75,
           ...
           -87.75,  -88.0, -88.25,  -88.5, -88.75,  -89.0, -89.25,  -89.5, -89.75,
            -90.0],
          dtype='float64', name='latitude', length=721))

  • longitude
    PandasIndex

    PandasIndex(Index([   0.0,   0.25,    0.5,   0.75,    1.0,   1.25,    1.5,   1.75,    2.0,
             2.25,
           ...
            357.5, 357.75,  358.0, 358.25,  358.5, 358.75,  359.0, 359.25,  359.5,
           359.75],
          dtype='float64', name='longitude', length=1440))

• Attributes: (12)

  GRIB_edition :

  2

  GRIB_centre :

  kwbc

  GRIB_centreDescription :

  US National Weather Service - NCEP

  GRIB_subCentre :

  0

  Conventions :

  CF-1.7

  institution :

  US National Weather Service - NCEP

  model :

  gfs

  product :

  pgrb2.0p25

  description :

  Global Forecast System

  remote_grib :

  https://noaa-gfs-bdp-pds.s3.amazonaws.com/gfs.20240101/00/atmos/gfs.t00z.pgrb2.0p25.f000

  local_grib :

  /home/blaylock/data/gfs/20240101/subset_6befcb50__gfs.t00z.pgrb2.0p25.f000

  searchString :

  :(?:TMP|DPT):2 m above

Now you can use all the fancy xarray bells and whitles to do stuff with this data.

ds.t2m.plot(cmap="Spectral_r", figsize=[8, 4])

<matplotlib.collections.QuadMesh at 0x7f7189f06e10>

# Compute the mean dew point for each latitude band
ds.d2m.mean(dim="longitude").plot()

[<matplotlib.lines.Line2D at 0x7f7189d073e0>]

ECMWF data

So far I’ve shown how to use Herbie to access GFS and HRRR data, but Herbie can discover many different model output files. The ECMWF publishes some open data from the Integrated Forecast System (IFS) publicly. The avaialble inventory for these GRIB files is slightly different than the inventories for the NCEP models.

• NCEP provides GRIB inventories produced by wgrib2;

• ECMWF provides GRIB inventories produced by eccodes.

This means the style of search string is going to be a little different when filtering ECMWF model GRIB fields.

H = Herbie("2024-03-15 18:00", model="ifs")
H.inventory()

✅ Found ┊ model=ifs ┊ product=oper ┊ 2024-Mar-15 18:00 UTC F00 ┊ GRIB2 @ azure-scda ┊ IDX @ azure-scda

	grib_message	start_byte	end_byte	range	reference_time	valid_time	step	param	levelist	levtype	number	domain	expver	class	type	stream	search_this
0	1	0	478649	0-478649	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	swvl1	NaN	sfc	NaN	g	0001	od	fc	scda	:swvl1:sfc:g:0001:od:fc:scda
1	2	478649	891087	478649-891087	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	r	500	pl	NaN	g	0001	od	fc	scda	:r:500:pl:g:0001:od:fc:scda
2	3	891087	1405385	891087-1405385	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	gh	925	pl	NaN	g	0001	od	fc	scda	:gh:925:pl:g:0001:od:fc:scda
3	4	1405385	2272665	1405385-2272665	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	10v	NaN	sfc	NaN	g	0001	od	fc	scda	:10v:sfc:g:0001:od:fc:scda
4	5	2272665	3021660	2272665-3021660	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	v	400	pl	NaN	g	0001	od	fc	scda	:v:400:pl:g:0001:od:fc:scda
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
136	137	103036703	104251475	103036703-104251475	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	vo	150	pl	NaN	g	0001	od	fc	scda	:vo:150:pl:g:0001:od:fc:scda
137	138	104251475	105602360	104251475-105602360	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	d	150	pl	NaN	g	0001	od	fc	scda	:d:150:pl:g:0001:od:fc:scda
138	139	105602360	106923274	105602360-106923274	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	d	50	pl	NaN	g	0001	od	fc	scda	:d:50:pl:g:0001:od:fc:scda
139	140	106923274	108049739	106923274-108049739	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	vo	50	pl	NaN	g	0001	od	fc	scda	:vo:50:pl:g:0001:od:fc:scda
140	141	108049739	108169539	108049739-108169539	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	asn	NaN	sfc	NaN	g	0001	od	fc	scda	:asn:sfc:g:0001:od:fc:scda

141 rows × 17 columns

Let’s look for all the temperature data at all pressure levels

H.inventory(r":t:\d+:pl")

	grib_message	start_byte	end_byte	range	reference_time	valid_time	step	param	levelist	levtype	number	domain	expver	class	type	stream	search_this
6	7	3740423	4353405	3740423-4353405	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	1000	pl	NaN	g	0001	od	fc	scda	:t:1000:pl:g:0001:od:fc:scda
13	14	8368341	9095588	8368341-9095588	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	200	pl	NaN	g	0001	od	fc	scda	:t:200:pl:g:0001:od:fc:scda
24	25	13008213	13545460	13008213-13545460	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	700	pl	NaN	g	0001	od	fc	scda	:t:700:pl:g:0001:od:fc:scda
25	26	13545460	14137300	13545460-14137300	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	300	pl	NaN	g	0001	od	fc	scda	:t:300:pl:g:0001:od:fc:scda
29	30	15476135	16086456	15476135-16086456	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	250	pl	NaN	g	0001	od	fc	scda	:t:250:pl:g:0001:od:fc:scda
32	33	17128847	17729306	17128847-17729306	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	850	pl	NaN	g	0001	od	fc	scda	:t:850:pl:g:0001:od:fc:scda
38	39	22299868	22920334	22299868-22920334	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	100	pl	NaN	g	0001	od	fc	scda	:t:100:pl:g:0001:od:fc:scda
40	41	23673370	24265086	23673370-24265086	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	400	pl	NaN	g	0001	od	fc	scda	:t:400:pl:g:0001:od:fc:scda
45	46	26894807	27499164	26894807-27499164	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	500	pl	NaN	g	0001	od	fc	scda	:t:500:pl:g:0001:od:fc:scda
46	47	27499164	28005180	27499164-28005180	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	600	pl	NaN	g	0001	od	fc	scda	:t:600:pl:g:0001:od:fc:scda
54	55	31814245	32414736	31814245-32414736	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	925	pl	NaN	g	0001	od	fc	scda	:t:925:pl:g:0001:od:fc:scda
62	63	37754947	38351787	37754947-38351787	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	150	pl	NaN	g	0001	od	fc	scda	:t:150:pl:g:0001:od:fc:scda
72	73	43926300	44556697	43926300-44556697	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	t	50	pl	NaN	g	0001	od	fc	scda	:t:50:pl:g:0001:od:fc:scda

Here is the 2-m temperature and dew point

H.inventory(r":2[t|d]")

	grib_message	start_byte	end_byte	range	reference_time	valid_time	step	param	levelist	levtype	number	domain	expver	class	type	stream	search_this
55	56	32414736	33106090	32414736-33106090	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	2d	NaN	sfc	NaN	g	0001	od	fc	scda	:2d:sfc:g:0001:od:fc:scda
64	65	38771272	39437754	38771272-39437754	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	2t	NaN	sfc	NaN	g	0001	od	fc	scda	:2t:sfc:g:0001:od:fc:scda

And here is 10-m U and V wind

H.inventory(r":10[u|v]")

	grib_message	start_byte	end_byte	range	reference_time	valid_time	step	param	levelist	levtype	number	domain	expver	class	type	stream	search_this
3	4	1405385	2272665	1405385-2272665	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	10v	NaN	sfc	NaN	g	0001	od	fc	scda	:10v:sfc:g:0001:od:fc:scda
58	59	34714064	35586123	34714064-35586123	2024-03-15 18:00:00	2024-03-15 18:00:00	0 days	10u	NaN	sfc	NaN	g	0001	od	fc	scda	:10u:sfc:g:0001:od:fc:scda

Use these same filters when downloading and opening with xarray.

Sometimes it takes some experientation to find exactly what you are looking for. But keep at it and you’ll get the hang of it.