{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# FastHerbie\n", "\n", "Often, data from several GRIB2 files is needed (range of datetimes and/or forecast lead time). ``FastHerbie()`` use multithreading to help you efficiently create multiple Herbie objects. Then you can download many files using multiprocessing. When you open the data with xarray, it will concatenate the DataSets for a range of model runs and forecast lead times.\n", "\n", "In this example, we will get the F00-F02 forecasts for each of the runs initialized between 00z-02z on January 1, 2022 (a total of 9 Herbie objects).\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from herbie import FastHerbie\n", "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# Create a range of dates\n", "DATES = pd.date_range(\n", " start=\"2022-03-01 00:00\",\n", " periods=3,\n", " freq=\"1H\",\n", ")\n", "\n", "# Create a range of forecast lead times\n", "fxx = range(0, 3)" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Make FastHerbie Object.\n", "FH = FastHerbie(DATES, model=\"hrrr\", fxx=fxx)\n", "FH" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "At it's core, ``FastHerbie`` uses multithreading to make a list of Herbie objects. The list of Herbie objects is stored in the `objects` property." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[\u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 00:00 UTC\u001b[92m F00\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 00:00 UTC\u001b[92m F01\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 00:00 UTC\u001b[92m F02\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 01:00 UTC\u001b[92m F00\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 01:00 UTC\u001b[92m F01\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 01:00 UTC\u001b[92m F02\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 02:00 UTC\u001b[92m F00\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 02:00 UTC\u001b[92m F01\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m,\n", " \u001b[48;2;255;255;255m\u001b[38;2;136;33;27m▌\u001b[0m\u001b[38;2;12;53;118m\u001b[48;2;240;234;210m▌\u001b[38;2;0;0;0m\u001b[1mHerbie\u001b[0m HRRR model \u001b[3msfc\u001b[0m product initialized \u001b[38;2;41;130;13m2022-Mar-01 02:00 UTC\u001b[92m F02\u001b[0m ┊ \u001b[38;2;255;153;0m\u001b[3msource=aws\u001b[0m]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "FH.objects" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You can download those Herbie objects as full files or subsets." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# Download full file\n", "FH.download()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Download subset\n", "FH.download(\"TMP:2 m\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "You can also read the data with xarray" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "👨🏻‍🏭 Created directory: [C:\\Users\\blaylock\\data\\hrrr\\20220301]\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:              (time: 3, step: 3, y: 1059, x: 1799)\n",
       "Coordinates:\n",
       "  * time                 (time) datetime64[ns] 2022-03-01 ... 2022-03-01T02:0...\n",
       "  * step                 (step) timedelta64[ns] 00:00:00 01:00:00 02:00:00\n",
       "    heightAboveGround    float64 2.0\n",
       "    latitude             (y, x) float64 21.14 21.15 21.15 ... 47.86 47.85 47.84\n",
       "    longitude            (y, x) float64 237.3 237.3 237.3 ... 299.0 299.0 299.1\n",
       "    valid_time           (step, time) datetime64[ns] 2022-03-01 ... 2022-03-0...\n",
       "Dimensions without coordinates: y, x\n",
       "Data variables:\n",
       "    t2m                  (step, time, y, x) float32 292.7 292.7 ... 263.4 263.4\n",
       "    gribfile_projection  object None\n",
       "Attributes:\n",
       "    GRIB_edition:            2\n",
       "    GRIB_centre:             kwbc\n",
       "    GRIB_centreDescription:  US National Weather Service - NCEP\n",
       "    GRIB_subCentre:          0\n",
       "    Conventions:             CF-1.7\n",
       "    institution:             US National Weather Service - NCEP\n",
       "    model:                   hrrr\n",
       "    product:                 sfc\n",
       "    description:             High-Resolution Rapid Refresh - CONUS\n",
       "    searchString:            TMP:2 m
" ], "text/plain": [ "\n", "Dimensions: (time: 3, step: 3, y: 1059, x: 1799)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2022-03-01 ... 2022-03-01T02:0...\n", " * step (step) timedelta64[ns] 00:00:00 01:00:00 02:00:00\n", " heightAboveGround float64 2.0\n", " latitude (y, x) float64 21.14 21.15 21.15 ... 47.86 47.85 47.84\n", " longitude (y, x) float64 237.3 237.3 237.3 ... 299.0 299.0 299.1\n", " valid_time (step, time) datetime64[ns] 2022-03-01 ... 2022-03-0...\n", "Dimensions without coordinates: y, x\n", "Data variables:\n", " t2m (step, time, y, x) float32 292.7 292.7 ... 263.4 263.4\n", " gribfile_projection object None\n", "Attributes:\n", " GRIB_edition: 2\n", " GRIB_centre: kwbc\n", " GRIB_centreDescription: US National Weather Service - NCEP\n", " GRIB_subCentre: 0\n", " Conventions: CF-1.7\n", " institution: US National Weather Service - NCEP\n", " model: hrrr\n", " product: sfc\n", " description: High-Resolution Rapid Refresh - CONUS\n", " searchString: TMP:2 m" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds = FH.xarray(\"TMP:2 m\", remove_grib=False)\n", "ds" ] } ], "metadata": { "kernelspec": { "display_name": "herbie-dev", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" }, "orig_nbformat": 4, "vscode": { "interpreter": { "hash": "ac511961bc6976a3d289cde8fc247e6b0e9298cf513cb762aa0a7df646d192ed" } } }, "nbformat": 4, "nbformat_minor": 2 }