⌚ Latest Data

Herbie can help you find the latest model data available with HerbieLatest and HerbieWait. These are simple helper functions that loops until Herbie finds data.

from herbie import HerbieLatest, HerbieWait

HerbieLatest

Herbie can find the latest model grid available. It does this by checking the for data in the most recent time, and then steps back in time a model cycle and checks for that. By default, if data is not found in four attempts, it will return an error.

H = HerbieLatest(model="hrrr", fxx=6)
H

✅ Found ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-19 19:00 UTC F06 ┊ GRIB2 @ aws ┊ IDX @ aws

▌▌Herbie HRRR model sfc product initialized 2024-Jan-19 19:00 UTC F06 ┊ source=aws

H = HerbieLatest(model="gfs")
H

✅ Found ┊ model=gfs ┊ product=pgrb2.0p25 ┊ 2024-Jan-19 18:00 UTC F00 ┊ GRIB2 @ aws ┊ IDX @ aws

▌▌Herbie GFS model pgrb2.0p25 product initialized 2024-Jan-19 18:00 UTC F00 ┊ source=aws

# This will error because there will never been
# a lead time of 100 hours.
H = HerbieLatest(model="hrrr", fxx=100)

💔 Did not find ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-19 19:00 UTC F100
💔 Did not find ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-19 18:00 UTC F100
💔 Did not find ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-19 17:00 UTC F100
💔 Did not find ┊ model=hrrr ┊ product=sfc ┊ 2024-Jan-19 16:00 UTC F100

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
Cell In[5], line 3
      1 # This will error because there will never been
      2 # a lead time of 100 hours.
----> 3 H = HerbieLatest(model="hrrr", fxx=100)

File ~/GITHUB/Herbie/herbie/latest.py:53, in HerbieLatest(model, priority, periods, **kwargs)
     50     if H.grib:
     51         return H
---> 53 raise TimeoutError(f"Herbie did not find data for the latest time: {H}")

TimeoutError: Herbie did not find data for the latest time: ║HERBIE╠ HRRR:sfc

HerbieWait

If you want to wait for model data to become available in real time, Herbie will use a while loop to wait for data.

In this example, I’ve changed the default wait time and interval just to demonstrate. (The error in this is expected.)

import pandas as pd

run = pd.Timestamp("now", tz="utc").floor('1h').replace(tzinfo=None)
H = HerbieWait(run=run, model="rap", product="awp130pgrb", wait_for="10s", check_interval="1s", fxx=0)

💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
Cell In[14], line 2
      1 run = pd.Timestamp("now", tz="utc").floor('1h').replace(tzinfo=None)
----> 2 H = HerbieWait(run=run, model="rap", product="awp130pgrb", wait_for="10s", check_interval="1s", fxx=0)

File ~/mapwall_dev/herbie-dev/herbie/latest.py:110, in HerbieWait(run, model, priority, wait_for, check_interval, **kwargs)
    108     # Error out if timeout is exceeded
    109     if (pd.Timestamp("now") - timer) >= pd.Timedelta(wait_for):
--> 110         raise TimeoutError(f"Herbie did not find data in time: {H}")
    112 return Herbie(run, model=model, priority=priority, **kwargs)

TimeoutError: Herbie did not find data in time: ║HERBIE╠ RAP:awp130pgrb

Here’s an example that takes a while to run, demonstrating that it works!

H = HerbieWait(run=run, model="rap", product="awp130pgrb", wait_for="2h", check_interval="120s", fxx=0)

💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
💔 Did not find ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00
✅ Found ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00 ┊ GRIB2 @ nomads ┊ IDX @ nomads
✅ Found ┊ model=rap ┊ product=awp130pgrb ┊ 2024-May-21 17:00 UTC F00 ┊ GRIB2 @ nomads ┊ IDX @ nomads

H

▌▌Herbie RAP model awp130pgrb product initialized 2024-May-21 17:00 UTC F00 ┊ source=nomads