fieldwork

Building a Target List From a Hail Event

How to turn a hail event into a list of addresses: report days, geometry, geocoding, point in polygon tests and provenance.

The event is a day, and the day may not be the one you remember

In a sales meeting a hail event is "the storm Tuesday night." In the public record it is a set of rows filed under a date, and that date follows a convention you did not pick. SPC groups its storm reports into a day that runs from 1200 UTC to 1159 UTC, which begins at 6 AM CST or 7 AM CDT depending on the time of year. So the report day labeled Tuesday holds everything from Tuesday morning through Wednesday morning local time. A storm that fires in the evening and is still dropping hail after midnight puts all of those reports on Tuesday, even though the homeowner who got hit after midnight will tell you it happened Wednesday.

Write the report day down in that convention before you do anything else. Every pull you make later keys off it, and a one day error puts you on a different storm or on no storm at all.

Write down a second thing too: these early reports are preliminary. SPC takes them from NWS Local Storm Reports, which are usually sent in near real time, and labels them preliminary and shows them as is. Reports arrive late, get corrected, and occasionally disappear. That is not a defect to engineer around. It is what a feed assembled during a storm looks like.

Get a geometry, not a place name

The second step separates a list from a guess. You need a shape with coordinates, and you need to know which kind, because the three kinds available mean three different things.

A warning polygon is a forecaster's drawing of the threatened area. Since the National Weather Service upgraded warning capabilities on October 1, 2007, warning polygons only cover the portion of the county actually threatened by the storm. Before that change, warnings were county based and covered the whole county regardless of where the storm sat. A polygon is tighter than a county and does not respect county lines. The same handout shows tornado warnings in effect for Hale, Bibb, Perry, Tuscaloosa, Jefferson and Shelby counties while the cities of Tuscaloosa, Birmingham and Calera sat outside the polygons and were not under a tornado warning. A polygon is also a forecast of threat, not a record of what fell.

A report point is a different object. It is a location where a person observed hail and told somebody. It carries no radius. It tells you a stone of some size hit that spot, and nothing about the house two streets over.

A radar derived footprint is the third kind. NCEI's Severe Weather Data Inventory carries NEXRAD Level-III hail signatures and storm structure among other datasets, and the page is direct about what they are: missing data does not mean no severe weather occurred, and much of the automatically derived data is radar based and represents probable rather than confirmed conditions. A radar footprint is the only one of the three that gives you area coverage, and it is an estimate.

Pick one, or combine them on purpose, but know which one you are holding. The rest of this process is mechanical. This step is the judgment.

Geocode the addresses you already have

Now you need coordinates for your address list. Two free options cover most of what a roofing company needs, and they behave differently.

The US Census Geocoder offers interactive and REST access, and its batch path takes up to 10,000 addresses at a time in CSV, XLS, XLSX, TXT or DAT format, with the caller choosing a benchmark and a vintage, where the vintage options depend on the benchmark chosen. Benchmark and vintage are not decoration. They pin which snapshot of Census geography you matched against, and they belong in your notes. The hard limit on this tool is coverage: it only geocodes addresses that are within the United States, Puerto Rico, and the U.S. Island Areas. For a domestic roofing list that is fine.

Nominatim, the OpenStreetMap geocoder, is the other common choice, and its usage policy is strict in ways that matter if you planned to loop over a list. The policy sets an absolute maximum of 1 request per second, applied per website or application rather than per user, and caps bulk scripts that run longer than a day or on a schedule at 4 requests per minute. It requires a valid HTTP Referer or User-Agent identifying your application and says plainly that stock User-Agents as set by http libraries will not do. It also states that bulk geocoding of larger amounts of data is not encouraged, allows small one time jobs on a single thread on one machine with no distributed scripts, and requires that results be cached on your side. Read that as a design instruction: cache every geocode you receive, keyed by the exact query string, and never geocode the same address twice.

The point in polygon test

With coordinates on one side and a geometry on the other, "which of these addresses was in the swath" is a point in polygon test. Any GIS library will do it. The GeoJSON conventions are where people lose a day.

A position is an array whose first two elements are longitude then latitude, with an optional third element for altitude in meters. Longitude first. Most people type latitude first, because that is how they say it out loud, and the result is a point that falls inside nothing.

Polygons are built from linear rings. Every ring needs at least four positions and must close, meaning its first and last positions hold identical values. Exterior rings run counterclockwise and holes run clockwise, by the right hand rule, though parsers are told not to reject polygons that ignore this for backward compatibility. The first ring is the exterior ring and any additional rings are holes. The default coordinate reference system is the WGS 84 datum in decimal degrees, and alternative coordinate reference systems were removed in that version of the spec.

Record where everything came from

The last step is quick and is the one most lists skip. For every list you build, record the source of the geometry, the source of the geocodes, and the date and time you pulled each one. Add the report day in the 1200 UTC convention, the benchmark and vintage if you used the Census geocoder, and the exact dataset name if you used a radar product.

The reason is that the record changes under you. Preliminary reports get revised. A radar derived dataset estimates probable conditions, not what landed on a given roof. If someone asks in month four why a particular address is on the list, "it was in the swath" is not an answer. "It was inside this polygon, pulled from this source on this date, and here is the geocode that put it there" is.

Three ways a list goes wrong

The first failure is geocode precision. An address that resolves to a street centroid or a ZIP centroid instead of a parcel can sit a long way from the actual roof, and near a polygon edge that is the difference between in and out. The fix is to read the match quality your geocoder returns, not just the coordinates, and to treat edge addresses as unknown instead of yes or no.

The second failure is coordinate order. If your polygon is stored latitude first and your points are longitude first, or the reverse, every test returns false and nothing errors. A silent empty result looks exactly like a storm that missed. Test the pipeline against one address you know was hit before you trust a zero.

The third failure is the list that was never built from a geometry at all. A list of every address in a county, or every address in a city, is not a target list. The polygon covers only the portion of the county actually threatened, and the Birmingham example shows named cities sitting outside polygons drawn across their own counties. A county list is mostly addresses that were not hit, and your crews pay for that in wasted time.

What a good list looks like

A good list is reproducible. The test is simple: hand your notes to someone else in the company and see whether they can rebuild the same list without talking to you. If they can, you have a record you can defend, reuse next season, and compare against the official data once it catches up. If they cannot, you have a spreadsheet.

Storm Stream is an API over the same public feeds: it builds swath geometry and answers which of a list of your own points fell inside it.